Anurag1734/cuda-error-resolution-analysis
07
1[2 {3 "post_stream": {4 "posts": [5 {6 "id": 144844,7 "name": "Peter Ham",8 "username": "Peter_Ham",9 "avatar_template": "/user_avatar/discuss.pytorch.org/peter_ham/{size}/15296_2.png",10 "created_at": "2019-11-08T06:52:12.036Z",11 "cooked": "<p>in order to use LSTM, you need a hidden state and a cell state, which is not provided in the first place. My question is how to you initialize the hidden state and the cell state for the first input? If it is randomly initialized then if I feed into the second input, the same initialization should also work to predict the next output. But it does not make sense to me that inputting different hidden/cell state would come up with the same output.</p>",12 "post_number": 1,13 "post_type": 1,14 "posts_count": 2,15 "updated_at": "2019-11-08T06:52:12.036Z",16 "reply_count": 0,17 "reply_to_post_number": null,18 "quote_count": 0,19 "incoming_link_count": 5474,20 "reads": 218,21 "readers_count": 217,22 "score": 27428.6,23 "yours": false,24 "topic_id": 60344,25 "topic_slug": "how-to-initialize-the-hidden-state-of-a-lstm",26 "display_username": "Peter Ham",27 "primary_group_name": null,28 "flair_name": null,29 "flair_url": null,30 "flair_bg_color": null,31 "flair_color": null,32 "flair_group_id": null,33 "badges_granted": [],34 "version": 1,35 "can_edit": false,36 "can_delete": false,37 "can_recover": false,38 "can_see_hidden_post": false,39 "can_wiki": false,40 "read": true,41 "user_title": null,42 "bookmarked": false,43 "actions_summary": [44 {45 "id": 2,46 "count": 147 }48 ],49 "moderator": false,50 "admin": false,51 "staff": false,52 "user_id": 2113,53 "hidden": false,54 "trust_level": 2,55 "deleted_at": null,56 "user_deleted": false,57 "edit_reason": null,58 "can_view_edit_history": true,59 "wiki": false,60 "post_url": "/t/how-to-initialize-the-hidden-state-of-a-lstm/60344/1",61 "can_accept_answer": false,62 "can_unaccept_answer": false,63 "accepted_answer": false,64 "topic_accepted_answer": null,65 "can_vote": false66 },67 {68 "id": 144855,69 "name": "Thomas V",70 "username": "tom",71 "avatar_template": "/user_avatar/discuss.pytorch.org/tom/{size}/3162_2.png",72 "created_at": "2019-11-08T08:23:37.013Z",73 "cooked": "<p>Even though that is in the context of BPTT, I think similar considerations apply as discussed the first item in:</p>\n<aside class=\"quote quote-modified\" data-post=\"2\" data-topic=\"58384\">\n <div class=\"title\">\n <div class=\"quote-controls\"></div>\n <img loading=\"lazy\" alt=\"\" width=\"24\" height=\"24\" src=\"https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/tom/48/3162_2.png\" class=\"avatar\">\n <a href=\"https://discuss.pytorch.org/t/initialization-of-first-hidden-state-in-lstm-and-truncated-bptt/58384/2\">Initialization of first hidden state in LSTM and truncated BPTT</a> \n </div>\n <blockquote>\n Yes, zero initial hiddenstate is standard so much so that it is the default in nn.LSTM if you don’t pass in a hidden state (rather than, e.g. throwing an error). Random initialization could also be used if zeros don’t work. Two basic ideas here: \n\nIf your hidden state evolution is “ergodic”, the state will move closer to some “steady distribution” anyways, so it doesn’t matter as much.\nYou want the initial hidden state handling to be somewhat consistent between training and inference.\nThe fan…\n </blockquote>\n</aside>\n\n<p>Best regards</p>\n<p>Thomas</p>",74 "post_number": 2,75 "post_type": 1,76 "posts_count": 2,77 "updated_at": "2019-11-08T08:23:37.013Z",78 "reply_count": 0,79 "reply_to_post_number": null,80 "quote_count": 0,81 "incoming_link_count": 33,82 "reads": 215,83 "readers_count": 214,84 "score": 208.0,85 "yours": false,86 "topic_id": 60344,87 "topic_slug": "how-to-initialize-the-hidden-state-of-a-lstm",88 "display_username": "Thomas V",89 "primary_group_name": null,90 "flair_name": null,91 "flair_url": null,92 "flair_bg_color": null,93 "flair_color": null,94 "flair_group_id": null,95 "badges_granted": [],96 "version": 1,97 "can_edit": false,98 "can_delete": false,99 "can_recover": false,100 "can_see_hidden_post": false,101 "can_wiki": false,102 "link_counts": [103 {104 "url": "https://discuss.pytorch.org/t/initialization-of-first-hidden-state-in-lstm-and-truncated-bptt/58384/2",105 "internal": true,106 "reflection": false,107 "title": "Initialization of first hidden state in LSTM and truncated BPTT",108 "clicks": 0109 }110 ],111 "read": true,112 "user_title": null,113 "bookmarked": false,114 "actions_summary": [],115 "moderator": false,116 "admin": false,117 "staff": false,118 "user_id": 616,119 "hidden": false,120 "trust_level": 2,121 "deleted_at": null,122 "user_deleted": false,123 "edit_reason": null,124 "can_view_edit_history": true,125 "wiki": false,126 "post_url": "/t/how-to-initialize-the-hidden-state-of-a-lstm/60344/2",127 "can_accept_answer": false,128 "can_unaccept_answer": false,129 "accepted_answer": false,130 "topic_accepted_answer": null131 }132 ],133 "stream": [134 144844,135 144855136 ]137 },138 "timeline_lookup": [139 [140 1,141 2179142 ]143 ],144 "suggested_topics": [145 {146 "fancy_title": "Accuracy remains same for epochs error while training large model on multiple GPUs",147 "id": 213073,148 "title": "Accuracy remains same for epochs error while training large model on multiple GPUs",149 "slug": "accuracy-remains-same-for-epochs-error-while-training-large-model-on-multiple-gpus",150 "posts_count": 1,151 "reply_count": 0,152 "highest_post_number": 1,153 "image_url": null,154 "created_at": "2024-11-17T10:58:18.888Z",155 "last_posted_at": "2024-11-17T10:58:18.947Z",156 "bumped": true,157 "bumped_at": "2024-11-17T10:58:18.947Z",158 "archetype": "regular",159 "unseen": false,160 "pinned": false,161 "unpinned": null,162 "visible": true,163 "closed": false,164 "archived": false,165 "bookmarked": null,166 "liked": null,167 "tags_descriptions": {},168 "like_count": 0,169 "views": 16,170 "category_id": 1,171 "featured_link": null,172 "has_accepted_answer": false,173 "posters": [174 {175 "extras": "latest single",176 "description": "Original Poster, Most Recent Poster",177 "user": {178 "id": 80963,179 "username": "eshanth_patyal",180 "name": "eshanth patyal",181 "avatar_template": "/user_avatar/discuss.pytorch.org/eshanth_patyal/{size}/72691_2.png",182 "trust_level": 0183 }184 }185 ]186 },187 {188 "fancy_title": "Broadcasting vs repeating",189 "id": 214419,190 "title": "Broadcasting vs repeating",191 "slug": "broadcasting-vs-repeating",192 "posts_count": 2,193 "reply_count": 0,194 "highest_post_number": 2,195 "image_url": null,196 "created_at": "2024-12-19T20:41:05.163Z",197 "last_posted_at": "2024-12-19T21:49:15.855Z",198 "bumped": true,199 "bumped_at": "2024-12-19T21:49:15.855Z",200 "archetype": "regular",201 "unseen": false,202 "pinned": false,203 "unpinned": null,204 "visible": true,205 "closed": false,206 "archived": false,207 "bookmarked": null,208 "liked": null,209 "tags_descriptions": {},210 "like_count": 1,211 "views": 165,212 "category_id": 1,213 "featured_link": null,214 "has_accepted_answer": false,215 "posters": [216 {217 "extras": null,218 "description": "Original Poster",219 "user": {220 "id": 72757,221 "username": "sherlock.h",222 "name": "",223 "avatar_template": "/user_avatar/discuss.pytorch.org/sherlock.h/{size}/62377_2.png",224 "trust_level": 1225 }226 },227 {228 "extras": "latest",229 "description": "Most Recent Poster",230 "user": {231 "id": 41396,232 "username": "soulitzer",233 "name": "",234 "avatar_template": "/letter_avatar_proxy/v4/letter/s/839c29/{size}.png",235 "trust_level": 2236 }237 }238 ]239 },240 {241 "fancy_title": "How I can improve validation loss in multi class text classification",242 "id": 214030,243 "title": "How I can improve validation loss in multi class text classification",244 "slug": "how-i-can-improve-validation-loss-in-multi-class-text-classification",245 "posts_count": 1,246 "reply_count": 0,247 "highest_post_number": 1,248 "image_url": null,249 "created_at": "2024-12-10T06:44:26.508Z",250 "last_posted_at": "2024-12-10T06:44:26.561Z",251 "bumped": true,252 "bumped_at": "2024-12-10T06:44:26.561Z",253 "archetype": "regular",254 "unseen": false,255 "pinned": false,256 "unpinned": null,257 "visible": true,258 "closed": false,259 "archived": false,260 "bookmarked": null,261 "liked": null,262 "tags_descriptions": {},263 "like_count": 0,264 "views": 32,265 "category_id": 1,266 "featured_link": null,267 "has_accepted_answer": false,268 "posters": [269 {270 "extras": "latest single",271 "description": "Original Poster, Most Recent Poster",272 "user": {273 "id": 81420,274 "username": "Gaurang_Gohel",275 "name": "Gaurang Gohel",276 "avatar_template": "/user_avatar/discuss.pytorch.org/gaurang_gohel/{size}/73797_2.png",277 "trust_level": 0278 }279 }280 ]281 },282 {283 "fancy_title": "Precision-ill is undefined while training Pytorch Models",284 "id": 218235,285 "title": "Precision-ill is undefined while training Pytorch Models",286 "slug": "precision-ill-is-undefined-while-training-pytorch-models",287 "posts_count": 1,288 "reply_count": 0,289 "highest_post_number": 1,290 "image_url": null,291 "created_at": "2025-03-25T10:39:35.467Z",292 "last_posted_at": "2025-03-25T10:39:35.509Z",293 "bumped": true,294 "bumped_at": "2025-03-25T15:22:02.738Z",295 "archetype": "regular",296 "unseen": false,297 "pinned": false,298 "unpinned": null,299 "visible": true,300 "closed": false,301 "archived": false,302 "bookmarked": null,303 "liked": null,304 "tags_descriptions": {},305 "like_count": 0,306 "views": 37,307 "category_id": 1,308 "featured_link": null,309 "has_accepted_answer": false,310 "posters": [311 {312 "extras": "latest single",313 "description": "Original Poster, Most Recent Poster",314 "user": {315 "id": 83401,316 "username": "koussayinsat12",317 "name": "GHAOUARI KOUSAI",318 "avatar_template": "/user_avatar/discuss.pytorch.org/koussayinsat12/{size}/76275_2.png",319 "trust_level": 1320 }321 }322 ]323 },324 {325 "fancy_title": "Distributed Tutorial works with mp.spawn but not torchrun",326 "id": 218381,327 "title": "Distributed Tutorial works with mp.spawn but not torchrun",328 "slug": "distributed-tutorial-works-with-mp-spawn-but-not-torchrun",329 "posts_count": 1,330 "reply_count": 0,331 "highest_post_number": 1,332 "image_url": "https://discuss.pytorch.org/uploads/default/original/3X/3/8/38bbe773a664165bad26af95d32fdf354409be61.png",333 "created_at": "2025-03-28T21:31:25.982Z",334 "last_posted_at": "2025-03-28T21:31:26.020Z",335 "bumped": true,336 "bumped_at": "2025-03-28T21:31:26.020Z",337 "archetype": "regular",338 "unseen": false,339 "pinned": false,340 "unpinned": null,341 "visible": true,342 "closed": false,343 "archived": false,344 "bookmarked": null,345 "liked": null,346 "tags_descriptions": {},347 "like_count": 0,348 "views": 26,349 "category_id": 1,350 "featured_link": null,351 "has_accepted_answer": false,352 "posters": [353 {354 "extras": "latest single",355 "description": "Original Poster, Most Recent Poster",356 "user": {357 "id": 83390,358 "username": "Marcelo_Sena",359 "name": "Marcelo Sena",360 "avatar_template": "/user_avatar/discuss.pytorch.org/marcelo_sena/{size}/76267_2.png",361 "trust_level": 0362 }363 }364 ]365 }366 ],367 "tags_descriptions": {},368 "fancy_title": "How to initialize the hidden state of a LSTM?",369 "id": 60344,370 "title": "How to initialize the hidden state of a LSTM?",371 "posts_count": 2,372 "created_at": "2019-11-08T06:52:11.983Z",373 "views": 7588,374 "reply_count": 0,375 "like_count": 1,376 "last_posted_at": "2019-11-08T08:23:37.013Z",377 "visible": true,378 "closed": false,379 "archived": false,380 "has_summary": false,381 "archetype": "regular",382 "slug": "how-to-initialize-the-hidden-state-of-a-lstm",383 "category_id": 1,384 "word_count": 125,385 "deleted_at": null,386 "user_id": 2113,387 "featured_link": null,388 "pinned_globally": false,389 "pinned_at": null,390 "pinned_until": null,391 "image_url": null,392 "slow_mode_seconds": 0,393 "draft": null,394 "draft_key": "topic_60344",395 "draft_sequence": null,396 "unpinned": null,397 "pinned": false,398 "current_post_number": 1,399 "highest_post_number": 2,400 "deleted_by": null,401 "actions_summary": [402 {403 "id": 4,404 "count": 0,405 "hidden": false,406 "can_act": false407 },408 {409 "id": 8,410 "count": 0,411 "hidden": false,412 "can_act": false413 },414 {415 "id": 10,416 "count": 0,417 "hidden": false,418 "can_act": false419 },420 {421 "id": 7,422 "count": 0,423 "hidden": false,424 "can_act": false425 }426 ],427 "chunk_size": 20,428 "bookmarked": false,429 "topic_timer": null,430 "message_bus_last_id": 0,431 "participant_count": 2,432 "show_read_indicator": false,433 "thumbnails": null,434 "slow_mode_enabled_until": null,435 "can_vote": false,436 "vote_count": 0,437 "user_voted": false,438 "discourse_zendesk_plugin_zendesk_id": null,439 "discourse_zendesk_plugin_zendesk_url": "https://your-url.zendesk.com/agent/tickets/",440 "details": {441 "can_edit": false,442 "notification_level": 1,443 "participants": [444 {445 "id": 616,446 "username": "tom",447 "name": "Thomas V",448 "avatar_template": "/user_avatar/discuss.pytorch.org/tom/{size}/3162_2.png",449 "post_count": 1,450 "primary_group_name": null,451 "flair_name": null,452 "flair_url": null,453 "flair_color": null,454 "flair_bg_color": null,455 "flair_group_id": null,456 "trust_level": 2457 },458 {459 "id": 2113,460 "username": "Peter_Ham",461 "name": "Peter Ham",462 "avatar_template": "/user_avatar/discuss.pytorch.org/peter_ham/{size}/15296_2.png",463 "post_count": 1,464 "primary_group_name": null,465 "flair_name": null,466 "flair_url": null,467 "flair_color": null,468 "flair_bg_color": null,469 "flair_group_id": null,470 "trust_level": 2471 }472 ],473 "created_by": {474 "id": 2113,475 "username": "Peter_Ham",476 "name": "Peter Ham",477 "avatar_template": "/user_avatar/discuss.pytorch.org/peter_ham/{size}/15296_2.png"478 },479 "last_poster": {480 "id": 616,481 "username": "tom",482 "name": "Thomas V",483 "avatar_template": "/user_avatar/discuss.pytorch.org/tom/{size}/3162_2.png"484 }485 },486 "bookmarks": []487 },488 {489 "post_stream": {490 "posts": [491 {492 "id": 140673,493 "name": "pedram",494 "username": "pedram1",495 "avatar_template": "/user_avatar/discuss.pytorch.org/pedram1/{size}/21032_2.png",496 "created_at": "2019-10-17T23:14:22.032Z",497 "cooked": "<p>Hello everyone</p>\n<p>I am trying to train my 1d conv autoencoder. In part of training the NN,I faced with the below error:</p>\n<p>//////////////////////////////////////////////////////////////////////////////////////////////////<br>\n“Using a target size (torch.Size([2048, 1, 94])) that is different to the input size (torch.Size([2048, 1, 93])). This will likely lead to incorrect results due to broadcasting. Please ensure they have the same size.<br>\nreturn F.mse_loss(input, target, reduction=self.reduction)”<br>\n//////////////////////////////////////////////////////////////////////////////////</p>\n<p>I am quite sure that my input tensor has the right size. because just before running the training loop, when ever I checked the size of that it gave me the right answer.<br>\nMy input size is (2040,1,94) which 2040 is batch number in every singel training, 1 is the input channel size and 94 is length of my signal.</p>\n<p>below is my training codes:</p>\n<p><span class=\"hashtag\">#training</span> loop<br>\n<br>\n<br>\nfor epoch in range(num_epochs):<br>\nfor data in dataloader:<br>\nimg = data</p>\n<pre><code> # ===================forward=====================\n output = model(img)\n loss = criterion(output, img)\n # ===================backward====================\n optimizer.zero_grad()\n loss.backward()\n optimizer.step()\n</code></pre>\n<p></p>",498 "post_number": 1,499 "post_type": 1,500 "posts_count": 12,501 "updated_at": "2019-10-17T23:14:43.075Z",502 "reply_count": 0,503 "reply_to_post_number": null,504 "quote_count": 0,505 "incoming_link_count": 1875,506 "reads": 52,507 "readers_count": 51,508 "score": 9375.4,509 "yours": false,510 "topic_id": 58527,511 "topic_slug": "mismatching-size-of-the-input-training-tensor-and-target",512 "display_username": "pedram",513 "primary_group_name": null,514 "flair_name": null,515 "flair_url": null,516 "flair_bg_color": null,517 "flair_color": null,518 "flair_group_id": null,519 "badges_granted": [],520 "version": 1,521 "can_edit": false,522 "can_delete": false,523 "can_recover": false,524 "can_see_hidden_post": false,525 "can_wiki": false,526 "read": true,527 "user_title": null,528 "bookmarked": false,529 "actions_summary": [],530 "moderator": false,531 "admin": false,532 "staff": false,533 "user_id": 23031,534 "hidden": false,535 "trust_level": 1,536 "deleted_at": null,537 "user_deleted": false,538 "edit_reason": null,539 "can_view_edit_history": true,540 "wiki": false,541 "post_url": "/t/mismatching-size-of-the-input-training-tensor-and-target/58527/1",542 "can_accept_answer": false,543 "can_unaccept_answer": false,544 "accepted_answer": false,545 "topic_accepted_answer": null,546 "can_vote": false547 },548 {549 "id": 140678,550 "name": "Alban D",551 "username": "albanD",552 "avatar_template": "/user_avatar/discuss.pytorch.org/alband/{size}/215_2.png",553 "created_at": "2019-10-17T23:39:22.946Z",554 "cooked": "<p>If you change your code to the following, what does it print?</p>\n<pre><code class=\"lang-auto\"> output = model(img)\n print(\"out\", output.size())\n print(\"img\", img.size())\n loss = criterion(output, img)\n</code></pre>",555 "post_number": 2,556 "post_type": 1,557 "posts_count": 12,558 "updated_at": "2019-10-17T23:39:22.946Z",559 "reply_count": 1,560 "reply_to_post_number": null,561 "quote_count": 0,562 "incoming_link_count": 4,563 "reads": 52,564 "readers_count": 51,565 "score": 50.4,566 "yours": false,567 "topic_id": 58527,568 "topic_slug": "mismatching-size-of-the-input-training-tensor-and-target",569 "display_username": "Alban D",570 "primary_group_name": null,571 "flair_name": null,572 "flair_url": null,573 "flair_bg_color": null,574 "flair_color": null,575 "flair_group_id": null,576 "badges_granted": [],577 "version": 1,578 "can_edit": false,579 "can_delete": false,580 "can_recover": false,581 "can_see_hidden_post": false,582 "can_wiki": false,583 "read": true,584 "user_title": "",585 "bookmarked": false,586 "actions_summary": [587 {588 "id": 2,589 "count": 1590 }591 ],592 "moderator": true,593 "admin": true,594 "staff": true,595 "user_id": 211,596 "hidden": false,597 "trust_level": 4,598 "deleted_at": null,599 "user_deleted": false,600 "edit_reason": null,601 "can_view_edit_history": true,602 "wiki": false,603 "post_url": "/t/mismatching-size-of-the-input-training-tensor-and-target/58527/2",604 "can_accept_answer": false,605 "can_unaccept_answer": false,606 "accepted_answer": false,607 "topic_accepted_answer": null608 },609 {610 "id": 140693,611 "name": "pedram",612 "username": "pedram1",613 "avatar_template": "/user_avatar/discuss.pytorch.org/pedram1/{size}/21032_2.png",614 "created_at": "2019-10-18T00:51:49.022Z",615 "cooked": "<aside class=\"quote no-group quote-modified\" data-username=\"albanD\" data-post=\"2\" data-topic=\"58527\">\n<div class=\"title\">\n<div class=\"quote-controls\"></div>\n<img loading=\"lazy\" alt=\"\" width=\"24\" height=\"24\" src=\"https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/alband/48/215_2.png\" class=\"avatar\"> albanD:</div>\n<blockquote>\n<p>print(“out”, output.size()) print(“img”, img.size())</p>\n</blockquote>\n</aside>\n<p>HI<br>\nThank you for your reply,<br>\nI did that. below you could see the results:</p>\n<p>out torch.Size([2048, 1, 93])<br>\nimg torch.Size([2048, 1, 94])</p>",616 "post_number": 3,617 "post_type": 1,618 "posts_count": 12,619 "updated_at": "2019-10-18T00:51:49.022Z",620 "reply_count": 1,621 "reply_to_post_number": 2,622 "quote_count": 1,623 "incoming_link_count": 5,624 "reads": 51,625 "readers_count": 50,626 "score": 40.2,627 "yours": false,628 "topic_id": 58527,629 "topic_slug": "mismatching-size-of-the-input-training-tensor-and-target",630 "display_username": "pedram",631 "primary_group_name": null,632 "flair_name": null,633 "flair_url": null,634 "flair_bg_color": null,635 "flair_color": null,636 "flair_group_id": null,637 "badges_granted": [],638 "version": 1,639 "can_edit": false,640 "can_delete": false,641 "can_recover": false,642 "can_see_hidden_post": false,643 "can_wiki": false,644 "read": true,645 "user_title": null,646 "bookmarked": false,647 "actions_summary": [],648 "moderator": false,649 "admin": false,650 "staff": false,651 "user_id": 23031,652 "hidden": false,653 "trust_level": 1,654 "deleted_at": null,655 "user_deleted": false,656 "edit_reason": null,657 "can_view_edit_history": true,658 "wiki": false,659 "post_url": "/t/mismatching-size-of-the-input-training-tensor-and-target/58527/3",660 "can_accept_answer": false,661 "can_unaccept_answer": false,662 "accepted_answer": false,663 "topic_accepted_answer": null664 },665 {666 "id": 140739,667 "name": "Alex Veuthey",668 "username": "alex.veuthey",669 "avatar_template": "/user_avatar/discuss.pytorch.org/alex.veuthey/{size}/12176_2.png",670 "created_at": "2019-10-18T06:42:13.507Z",671 "cooked": "<p>That looks like your model does not output the correct size… It could happen in many places, I’d recommend printing the shape of internal variables in the <code>forward</code> method of your network, so that you can pinpoint the exact location that introduces the error and debug that.</p>\n<p>It would also help if you can share the code of your architecture! But it would be understandable if you can’t <img src=\"https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9\" title=\":slight_smile:\" class=\"emoji\" alt=\":slight_smile:\"></p>",672 "post_number": 4,673 "post_type": 1,674 "posts_count": 12,675 "updated_at": "2019-10-18T06:42:13.507Z",676 "reply_count": 1,677 "reply_to_post_number": 3,678 "quote_count": 0,679 "incoming_link_count": 2,680 "reads": 44,681 "readers_count": 43,682 "score": 38.8,683 "yours": false,684 "topic_id": 58527,685 "topic_slug": "mismatching-size-of-the-input-training-tensor-and-target",686 "display_username": "Alex Veuthey",687 "primary_group_name": null,688 "flair_name": null,689 "flair_url": null,690 "flair_bg_color": null,691 "flair_color": null,692 "flair_group_id": null,693 "badges_granted": [],694 "version": 1,695 "can_edit": false,696 "can_delete": false,697 "can_recover": false,698 "can_see_hidden_post": false,699 "can_wiki": false,700 "read": true,701 "user_title": null,702 "reply_to_user": {703 "id": 23031,704 "username": "pedram1",705 "name": "pedram",706 "avatar_template": "/user_avatar/discuss.pytorch.org/pedram1/{size}/21032_2.png"707 },708 "bookmarked": false,709 "actions_summary": [710 {711 "id": 2,712 "count": 1713 }714 ],715 "moderator": false,716 "admin": false,717 "staff": false,718 "user_id": 13993,719 "hidden": false,720 "trust_level": 2,721 "deleted_at": null,722 "user_deleted": false,723 "edit_reason": null,724 "can_view_edit_history": true,725 "wiki": false,726 "post_url": "/t/mismatching-size-of-the-input-training-tensor-and-target/58527/4",727 "can_accept_answer": false,728 "can_unaccept_answer": false,729 "accepted_answer": false,730 "topic_accepted_answer": null731 },732 {733 "id": 140887,734 "name": "pedram",735 "username": "pedram1",736 "avatar_template": "/user_avatar/discuss.pytorch.org/pedram1/{size}/21032_2.png",737 "created_at": "2019-10-18T21:54:15.918Z",738 "cooked": "<p>Hi<br>\nThank you for your reply.<br>\nMy codes are not really something important <img src=\"https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9\" title=\":slight_smile:\" class=\"emoji\" alt=\":slight_smile:\"><br>\nBelow you could be able to see them all:<br>\n//////////////////////////////////////////////////////////////////////////////////<br>\n<span class=\"hashtag\">#Autoencoder</span> and Autodecoder conv layer…<br>\nclass Autoencoder(nn.Module):</p>\n<p>def <strong>init</strong>(self):<br>\nsuper(Autoencoder,self).<strong>init</strong>()</p>\n<pre><code> self.encoder = nn.Sequential(\n nn.Conv1d(1,5,kernel_size=5, stride=2),\n nn.ReLU(True),\n nn.Conv1d(5,10,kernel_size=5, stride = 2),\n nn.ReLU(True),\n nn.Conv1d(10,20,kernel_size=5, stride = 2),\n nn.ReLU(True))\n self.decoder = nn.Sequential( \n nn.ConvTranspose1d(20,10,kernel_size=5, stride = 2),\n nn.ReLU(True),\n nn.ConvTranspose1d(10,5,kernel_size=5, stride = 2),\n nn.ReLU(True),\n nn.ConvTranspose1d(5,1,kernel_size=5, stride = 2),\n nn.Sigmoid())\n</code></pre>\n<p>def forward(self,x):<br>\nx = self.encoder(x)<br>\nx = self.decoder(x)<br>\nreturn x<br>\n//////////////////////////////////////////////////////////////////<br>\nnum_epochs = 32<br>\nbatch_size = 2048<br>\nlearning_rate = 1e-3<br>\n////////////////////////////////////////////////////////////<br>\nmodel = Autoencoder().cpu()<br>\ncriterion = nn.MSELoss()<br>\noptimizer = torch.optim.Adam(model.parameters(), lr=learning_rate,<br>\nweight_decay=1e-5)<br>\ndataloader = DataLoader(data_pixel2, batch_size=batch_size, shuffle=True)<br>\n///////////////////////////////////////////////////////////////////////////////</p>\n<p><span class=\"hashtag\">#training</span> loop<br>\n<br>\n<br>\nfor epoch in range(num_epochs):<br>\nfor data in dataloader:<br>\nimg = data</p>\n<pre><code> # ===================forward=====================\n output = model(img)\n print(\"out\", output.size())\n print(\"img\", img.size())\n loss = criterion(output, img)\n # ===================backward====================\n optimizer.zero_grad()\n loss.backward()\n optimizer.step()\n# ===================log========================\nprint('epoch [{}/{}], loss:{:.4f}'\n .format(epoch+1, num_epochs, loss.data[0]))\nif epoch % 10 == 0:\n pic = to_img(output.cpu().data)\n save_image(pic, './dc_img/image_{}.png'.format(epoch))\n</code></pre>\n<p>///////////////////////////////////////////////</p>\n<p>Regards,</p>",739 "post_number": 5,740 "post_type": 1,741 "posts_count": 12,742 "updated_at": "2019-10-18T21:54:15.918Z",743 "reply_count": 1,744 "reply_to_post_number": 4,745 "quote_count": 0,746 "incoming_link_count": 18,747 "reads": 41,748 "readers_count": 40,749 "score": 103.2,750 "yours": false,751 "topic_id": 58527,752 "topic_slug": "mismatching-size-of-the-input-training-tensor-and-target",753 "display_username": "pedram",754 "primary_group_name": null,755 "flair_name": null,756 "flair_url": null,757 "flair_bg_color": null,758 "flair_color": null,759 "flair_group_id": null,760 "badges_granted": [],761 "version": 1,762 "can_edit": false,763 "can_delete": false,764 "can_recover": false,765 "can_see_hidden_post": false,766 "can_wiki": false,767 "read": true,768 "user_title": null,769 "reply_to_user": {770 "id": 13993,771 "username": "alex.veuthey",772 "name": "Alex Veuthey",773 "avatar_template": "/user_avatar/discuss.pytorch.org/alex.veuthey/{size}/12176_2.png"774 },775 "bookmarked": false,776 "actions_summary": [],777 "moderator": false,778 "admin": false,779 "staff": false,780 "user_id": 23031,781 "hidden": false,782 "trust_level": 1,783 "deleted_at": null,784 "user_deleted": false,785 "edit_reason": null,786 "can_view_edit_history": true,787 "wiki": false,788 "post_url": "/t/mismatching-size-of-the-input-training-tensor-and-target/58527/5",789 "can_accept_answer": false,790 "can_unaccept_answer": false,791 "accepted_answer": false,792 "topic_accepted_answer": null793 },794 {795 "id": 141516,796 "name": "Alex Veuthey",797 "username": "alex.veuthey",798 "avatar_template": "/user_avatar/discuss.pytorch.org/alex.veuthey/{size}/12176_2.png",799 "created_at": "2019-10-22T06:47:30.758Z",800 "cooked": "<p>I think I know where the error comes from…</p>\n<p>Your convolutions have a stride of 2. This means roughly (I hope I’m right) that each conv layer divides your input size by 2 (and removes a few elements on the borders due to no padding by default in the conv layer).</p>\n<p>By printing the shape of the output of each layer, which you can do for the encoder with the following (the same with the decoder):</p>\n<pre><code class=\"lang-python\">for module in self.encoder:\n x = module(x)\n print(x.shape)\n</code></pre>\n<p>You will see that the shape follows the following sequence: <code>[94, 45, 21, 9]</code> in the encoder, but the decoder gives <code>[9, 21, 45, 93]</code>.</p>\n<p>Now, I’m not familiar with the autoencoder architecture, specifically with how you need to setup your input sizes and kernel sizes for correct downsizing and upsizing… So you’ll need to check some documentation on that, I’m afraid this is the furthest I can help you.</p>\n<p>I’ve just tried quickly changing the kernel size of the outermost convolutions (so the first conv in the encoder and the last conv transpose of the decoder) to 6 instead of 5, and this resulted in an output size of 94, but I’m not sure this is what you want or if it’s even valid…</p>\n<p>P.S: if you format your code properly (by inserting it in a code block enclosed in two groups of three backticks `), it would be much easier to read it and help you! Copy-pasting the code directly failed, because the <code>__init__</code> was changed to <strong>init</strong> due to markdown’s formatting shortcuts.</p>",801 "post_number": 6,802 "post_type": 1,803 "posts_count": 12,804 "updated_at": "2019-10-22T06:47:30.758Z",805 "reply_count": 0,806 "reply_to_post_number": 5,807 "quote_count": 0,808 "incoming_link_count": 10,809 "reads": 34,810 "readers_count": 33,811 "score": 71.8,812 "yours": false,813 "topic_id": 58527,814 "topic_slug": "mismatching-size-of-the-input-training-tensor-and-target",815 "display_username": "Alex Veuthey",816 "primary_group_name": null,817 "flair_name": null,818 "flair_url": null,819 "flair_bg_color": null,820 "flair_color": null,821 "flair_group_id": null,822 "badges_granted": [],823 "version": 1,824 "can_edit": false,825 "can_delete": false,826 "can_recover": false,827 "can_see_hidden_post": false,828 "can_wiki": false,829 "read": true,830 "user_title": null,831 "reply_to_user": {832 "id": 23031,833 "username": "pedram1",834 "name": "pedram",835 "avatar_template": "/user_avatar/discuss.pytorch.org/pedram1/{size}/21032_2.png"836 },837 "bookmarked": false,838 "actions_summary": [839 {840 "id": 2,841 "count": 1842 }843 ],844 "moderator": false,845 "admin": false,846 "staff": false,847 "user_id": 13993,848 "hidden": false,849 "trust_level": 2,850 "deleted_at": null,851 "user_deleted": false,852 "edit_reason": null,853 "can_view_edit_history": true,854 "wiki": false,855 "post_url": "/t/mismatching-size-of-the-input-training-tensor-and-target/58527/6",856 "can_accept_answer": false,857 "can_unaccept_answer": false,858 "accepted_answer": false,859 "topic_accepted_answer": null860 },861 {862 "id": 141904,863 "name": "pedram",864 "username": "pedram1",865 "avatar_template": "/user_avatar/discuss.pytorch.org/pedram1/{size}/21032_2.png",866 "created_at": "2019-10-23T18:15:41.884Z",867 "cooked": "<p>HI<br>\nThank you for your clear explanation. My problem actually solved, however when I run the training loop on my network loss of my NN was about 1088392064.000000 …</p>\n<p>Do you have any idea ???</p>",868 "post_number": 7,869 "post_type": 1,870 "posts_count": 12,871 "updated_at": "2019-10-23T18:15:41.884Z",872 "reply_count": 1,873 "reply_to_post_number": null,874 "quote_count": 0,875 "incoming_link_count": 1,876 "reads": 33,877 "readers_count": 32,878 "score": 16.6,879 "yours": false,880 "topic_id": 58527,881 "topic_slug": "mismatching-size-of-the-input-training-tensor-and-target",882 "display_username": "pedram",883 "primary_group_name": null,884 "flair_name": null,885 "flair_url": null,886 "flair_bg_color": null,887 "flair_color": null,888 "flair_group_id": null,889 "badges_granted": [],890 "version": 1,891 "can_edit": false,892 "can_delete": false,893 "can_recover": false,894 "can_see_hidden_post": false,895 "can_wiki": false,896 "read": true,897 "user_title": null,898 "bookmarked": false,899 "actions_summary": [],900 "moderator": false,901 "admin": false,902 "staff": false,903 "user_id": 23031,904 "hidden": false,905 "trust_level": 1,906 "deleted_at": null,907 "user_deleted": false,908 "edit_reason": null,909 "can_view_edit_history": true,910 "wiki": false,911 "post_url": "/t/mismatching-size-of-the-input-training-tensor-and-target/58527/7",912 "can_accept_answer": false,913 "can_unaccept_answer": false,914 "accepted_answer": false,915 "topic_accepted_answer": null916 },917 {918 "id": 141972,919 "name": "Alex Veuthey",920 "username": "alex.veuthey",921 "avatar_template": "/user_avatar/discuss.pytorch.org/alex.veuthey/{size}/12176_2.png",922 "created_at": "2019-10-24T06:03:03.988Z",923 "cooked": "<p>Is that the training loss? Does it start at this point or does it increase and stay there? Does it explode after that point?</p>\n<p>That can be many things, but try reducing the learning rate first, and see how that goes.</p>",924 "post_number": 8,925 "post_type": 1,926 "posts_count": 12,927 "updated_at": "2019-10-24T06:03:03.988Z",928 "reply_count": 1,929 "reply_to_post_number": 7,930 "quote_count": 0,931 "incoming_link_count": 1,932 "reads": 30,933 "readers_count": 29,934 "score": 16.0,935 "yours": false,936 "topic_id": 58527,937 "topic_slug": "mismatching-size-of-the-input-training-tensor-and-target",938 "display_username": "Alex Veuthey",939 "primary_group_name": null,940 "flair_name": null,941 "flair_url": null,942 "flair_bg_color": null,943 "flair_color": null,944 "flair_group_id": null,945 "badges_granted": [],946 "version": 1,947 "can_edit": false,948 "can_delete": false,949 "can_recover": false,950 "can_see_hidden_post": false,951 "can_wiki": false,952 "read": true,953 "user_title": null,954 "reply_to_user": {955 "id": 23031,956 "username": "pedram1",957 "name": "pedram",958 "avatar_template": "/user_avatar/discuss.pytorch.org/pedram1/{size}/21032_2.png"959 },960 "bookmarked": false,961 "actions_summary": [],962 "moderator": false,963 "admin": false,964 "staff": false,965 "user_id": 13993,966 "hidden": false,967 "trust_level": 2,968 "deleted_at": null,969 "user_deleted": false,970 "edit_reason": null,971 "can_view_edit_history": true,972 "wiki": false,973 "post_url": "/t/mismatching-size-of-the-input-training-tensor-and-target/58527/8",974 "can_accept_answer": false,975 "can_unaccept_answer": false,976 "accepted_answer": false,977 "topic_accepted_answer": null978 },979 {980 "id": 142346,981 "name": "pedram",982 "username": "pedram1",983 "avatar_template": "/user_avatar/discuss.pytorch.org/pedram1/{size}/21032_2.png",984 "created_at": "2019-10-25T19:24:33.442Z",985 "cooked": "<p>Yes it is training loss as far as I understand.<br>\nBelow you can see my codes in order to calculate the loss:<br>\n<span class=\"hashtag\">#training</span> loop</p>\n<p>for epoch in range(num_epochs):<br>\nfor data in dataloader:<br>\nimg = data<br>\nimg = Variable(img)</p>\n<pre><code>output = model (img)\nloss = criterion (output,img)\n\noptimizer.zero_grad()\nloss.backward()\noptimizer.step()\n\nif epoch%10 == 1:\n print ('\\r Train Epoch : {}/{} \\tLoss : {:.6f}'.format (epoch+1,num_epochs,loss))\n</code></pre>\n<p>I also wrote you down the loss of the last epoch :<br>\nTrain Epoch : 72/80 \tLoss : 1088451712.000000<br>\nTrain Epoch : 72/80 \tLoss : 1088443520.000000<br>\nTrain Epoch : 72/80 \tLoss : 1088414208.000000<br>\nTrain Epoch : 72/80 \tLoss : 1088391808.000000<br>\nTrain Epoch : 72/80 \tLoss : 1088355584.000000<br>\nTrain Epoch : 72/80 \tLoss : 1088334464.000000</p>",986 "post_number": 9,987 "post_type": 1,988 "posts_count": 12,989 "updated_at": "2019-10-25T19:24:33.442Z",990 "reply_count": 1,991 "reply_to_post_number": 8,992 "quote_count": 0,993 "incoming_link_count": 2,994 "reads": 29,995 "readers_count": 28,996 "score": 20.8,997 "yours": false,998 "topic_id": 58527,999 "topic_slug": "mismatching-size-of-the-input-training-tensor-and-target",1000 "display_username": "pedram",1001 "primary_group_name": null,1002 "flair_name": null,1003 "flair_url": null,1004 "flair_bg_color": null,1005 "flair_color": null,1006 "flair_group_id": null,1007 "badges_granted": [],1008 "version": 1,1009 "can_edit": false,1010 "can_delete": false,1011 "can_recover": false,1012 "can_see_hidden_post": false,1013 "can_wiki": false,1014 "read": true,1015 "user_title": null,1016 "reply_to_user": {1017 "id": 13993,1018 "username": "alex.veuthey",1019 "name": "Alex Veuthey",1020 "avatar_template": "/user_avatar/discuss.pytorch.org/alex.veuthey/{size}/12176_2.png"1021 },1022 "bookmarked": false,1023 "actions_summary": [],1024 "moderator": false,1025 "admin": false,1026 "staff": false,1027 "user_id": 23031,1028 "hidden": false,1029 "trust_level": 1,1030 "deleted_at": null,1031 "user_deleted": false,1032 "edit_reason": null,1033 "can_view_edit_history": true,1034 "wiki": false,1035 "post_url": "/t/mismatching-size-of-the-input-training-tensor-and-target/58527/9",1036 "can_accept_answer": false,1037 "can_unaccept_answer": false,1038 "accepted_answer": false,1039 "topic_accepted_answer": null1040 },1041 {1042 "id": 142800,1043 "name": "Alex Veuthey",1044 "username": "alex.veuthey",1045 "avatar_template": "/user_avatar/discuss.pytorch.org/alex.veuthey/{size}/12176_2.png",1046 "created_at": "2019-10-29T07:16:10.614Z",1047 "cooked": "<p>I see.</p>\n<p>Well the MSELoss can be quite large due to its nature, but this is still very high… It also depends on the nature of your problem (so your <code>X</code> and <code>y</code> values).</p>\n<p>Also, does it start at the same values in the first epoch? This might indicate that your network is not learning anything.</p>",1048 "post_number": 10,1049 "post_type": 1,1050 "posts_count": 12,1051 "updated_at": "2019-10-29T07:16:10.614Z",1052 "reply_count": 1,1053 "reply_to_post_number": 9,1054 "quote_count": 0,1055 "incoming_link_count": 2,1056 "reads": 23,1057 "readers_count": 22,1058 "score": 34.6,1059 "yours": false,1060 "topic_id": 58527,1061 "topic_slug": "mismatching-size-of-the-input-training-tensor-and-target",1062 "display_username": "Alex Veuthey",1063 "primary_group_name": null,1064 "flair_name": null,1065 "flair_url": null,1066 "flair_bg_color": null,1067 "flair_color": null,1068 "flair_group_id": null,1069 "badges_granted": [],1070 "version": 1,1071 "can_edit": false,1072 "can_delete": false,1073 "can_recover": false,1074 "can_see_hidden_post": false,1075 "can_wiki": false,1076 "read": true,1077 "user_title": null,1078 "reply_to_user": {1079 "id": 23031,1080 "username": "pedram1",1081 "name": "pedram",1082 "avatar_template": "/user_avatar/discuss.pytorch.org/pedram1/{size}/21032_2.png"1083 },1084 "bookmarked": false,1085 "actions_summary": [1086 {1087 "id": 2,1088 "count": 11089 }1090 ],1091 "moderator": false,1092 "admin": false,1093 "staff": false,1094 "user_id": 13993,1095 "hidden": false,1096 "trust_level": 2,1097 "deleted_at": null,1098 "user_deleted": false,1099 "edit_reason": null,1100 "can_view_edit_history": true,1101 "wiki": false,1102 "post_url": "/t/mismatching-size-of-the-input-training-tensor-and-target/58527/10",1103 "can_accept_answer": false,1104 "can_unaccept_answer": false,1105 "accepted_answer": false,1106 "topic_accepted_answer": null1107 },1108 {1109 "id": 144740,1110 "name": "pedram",1111 "username": "pedram1",1112 "avatar_template": "/user_avatar/discuss.pytorch.org/pedram1/{size}/21032_2.png",1113 "created_at": "2019-11-07T17:14:06.303Z",1114 "cooked": "<p>Hi Alex<br>\nSorry for late replying back.<br>\nm.mmmm you are right.<br>\nActually I realized that I did not normalized my data. Do you have any idea that how I can do it by using data loader module ? I tried to use below code, however I don’t have good feeling about that. I believe that there might be a way of normalizing in pytorch rather than do it by for loops<br>\n//////////////////////////////////////////////////////////////////////////////////////////////<br>\n<span class=\"hashtag\">#again</span> reshape the data file<br>\nnew_data = np.empty((256,256,94))<br>\nfor i in range (256) :<br>\nfor j in range (256) :<br>\nfor k in range (94) :</p>\n<pre><code> new_data[i,j,k] = (data[k,i,j]-(np.mean(data[:,i,j])))/((np.amax(data[:,i,j]))-(np.amin([data[:,i,j]])))\n</code></pre>\n<p>////////////////////////////////////////////////////////////////////////////////////////</p>\n<p>Thank you</p>",1115 "post_number": 11,1116 "post_type": 1,1117 "posts_count": 12,1118 "updated_at": "2019-11-07T17:14:06.303Z",1119 "reply_count": 1,1120 "reply_to_post_number": 10,1121 "quote_count": 0,1122 "incoming_link_count": 7,1123 "reads": 21,1124 "readers_count": 20,1125 "score": 44.2,1126 "yours": false,1127 "topic_id": 58527,1128 "topic_slug": "mismatching-size-of-the-input-training-tensor-and-target",1129 "display_username": "pedram",1130 "primary_group_name": null,1131 "flair_name": null,1132 "flair_url": null,1133 "flair_bg_color": null,1134 "flair_color": null,1135 "flair_group_id": null,1136 "badges_granted": [],1137 "version": 1,1138 "can_edit": false,1139 "can_delete": false,1140 "can_recover": false,1141 "can_see_hidden_post": false,1142 "can_wiki": false,1143 "read": true,1144 "user_title": null,1145 "reply_to_user": {1146 "id": 13993,1147 "username": "alex.veuthey",1148 "name": "Alex Veuthey",1149 "avatar_template": "/user_avatar/discuss.pytorch.org/alex.veuthey/{size}/12176_2.png"1150 },1151 "bookmarked": false,1152 "actions_summary": [],1153 "moderator": false,1154 "admin": false,1155 "staff": false,1156 "user_id": 23031,1157 "hidden": false,1158 "trust_level": 1,1159 "deleted_at": null,1160 "user_deleted": false,1161 "edit_reason": null,1162 "can_view_edit_history": true,1163 "wiki": false,1164 "post_url": "/t/mismatching-size-of-the-input-training-tensor-and-target/58527/11",1165 "can_accept_answer": false,1166 "can_unaccept_answer": false,1167 "accepted_answer": false,1168 "topic_accepted_answer": null1169 },1170 {1171 "id": 144847,1172 "name": "Alex Veuthey",1173 "username": "alex.veuthey",1174 "avatar_template": "/user_avatar/discuss.pytorch.org/alex.veuthey/{size}/12176_2.png",1175 "created_at": "2019-11-08T07:05:43.852Z",1176 "cooked": "<p>Indeed, this is far from efficient, and good thinking for the normalization, it slipped my mind!</p>\n<p>I’m not sure if this is the right way for your kind of data (I’ve only worked on images), but in general, what you want to do is compute your statistics in advance, on the whole training set (do not include validation or test sets, as this would introduce some bias), and then normalize input data in the dataset (not the data loader!) as it gets called, using your previously computed stats.</p>",1177 "post_number": 12,1178 "post_type": 1,1179 "posts_count": 12,1180 "updated_at": "2019-11-08T07:05:43.852Z",1181 "reply_count": 0,1182 "reply_to_post_number": 11,1183 "quote_count": 0,1184 "incoming_link_count": 1,1185 "reads": 21,1186 "readers_count": 20,1187 "score": 24.2,1188 "yours": false,1189 "topic_id": 58527,1190 "topic_slug": "mismatching-size-of-the-input-training-tensor-and-target",1191 "display_username": "Alex Veuthey",1192 "primary_group_name": null,1193 "flair_name": null,1194 "flair_url": null,1195 "flair_bg_color": null,1196 "flair_color": null,1197 "flair_group_id": null,1198 "badges_granted": [],1199 "version": 1,1200 "can_edit": false,