Anurag1734/cuda-error-resolution-analysis
07
1[2 {3 "post_stream": {4 "posts": [5 {6 "id": 204075,7 "name": "BRETT BYRNES",8 "username": "btb",9 "avatar_template": "/letter_avatar_proxy/v4/letter/b/9fc348/{size}.png",10 "created_at": "2020-06-17T21:18:08.678Z",11 "cooked": "<p>Getting the hang of PyTorch by doing a simple linear regression.</p>\n<p>When I use my own “custom” linear regression formula, I get fine results without changing my y_tensor (from a numpy array).</p>\n<p>When I use the built in PyTorch linear regression class, I have to pass “y_tensor = y_tensor.unsqueeze(-1)” other wise I get the error:</p>\n<pre><code class=\"lang-auto\">/usr/local/lib/python3.6/dist-packages/torch/nn/modules/loss.py:432: UserWarning: Using a target size (torch.Size([824])) that is different to the input size (torch.Size([824, 1])). This will likely lead to incorrect results due to broadcasting. Please ensure they have the same size.\n return F.mse_loss(input, target, reduction=self.reduction)\n</code></pre>\n<p>Code for not having to unsqueeze…</p>\n<pre><code class=\"lang-auto\">X_tensor = torch.from_numpy(X_train)\nX_tensor = X_tensor.float()\ny_tensor = torch.tensor(y_train.values,dtype=torch.float)\n\ndep_vars = X_tensor.shape[1]\n\nweights = torch.rand(dep_vars,requires_grad=True,dtype=torch.float)\nbias = torch.rand(1,requires_grad=True,dtype=torch.float)\n\nnum_iterations = 1000\nlearning_rate = 0.1\n\ndef model(X): # Model function of the form y = w*X + b\n return torch.mv(X_tensor, weights) + bias \n\nloss_fn = torch.nn.MSELoss() # Loss function MSE from PyTorch\n# Optimizer using my weights and bias Tensor from PyTorch\noptimizer = torch.optim.SGD([weights,bias],lr=learning_rate) \n\nfor epoch in range(0,num_iterations): # Loop Through\n predicted = model(X_tensor) # Compute predicted values\n loss = loss_fn(predicted, y_tensor) # Compute the loss\n if epoch % (num_iterations/10) == 0:\n print('loss at step ', ': ', loss)\n loss.backward() # Gradient of loss through backward propogation\n optimizer.step() # Optimizer update the weights\n optimizer.zero_grad() # Optimizer zero out the gradient for next step\n\nprint('weights: ', weights) # Print final weights and biases\nprint('bias: ', bias)\n</code></pre>\n<p>Code requiring unsqueeze…</p>\n<pre><code class=\"lang-auto\">X_tensor = torch.from_numpy(X_train)\nX_tensor = X_tensor.float()\ny_tensor = torch.tensor(y_train.values,dtype=torch.float)\ny_tensor = y_tensor.unsqueeze(-1)\n\ndep_vars = X_tensor.shape[1]\n\nlearning_rate = .1\nnum_iterations = 1000\n\nmodel = torch.nn.Linear(dep_vars, 1, bias=True)\nloss_fn = torch.nn.MSELoss()\noptimizer = torch.optim.SGD(model.parameters(), lr = learning_rate)\n\nfor epoch in range(0,num_iterations):\n\n pred = model(X_tensor) # Make predictions of Linear Model\n loss = loss_fn(pred, y_tensor) # Calcualte Loss\n loss.backward() # Backward pass\n optimizer.step() # Update weights \n optimizer.zero_grad() # Zero out gradient\n\n if epoch % (num_iterations / 10) == 0:\n print(f'{epoch} Loss: {loss}')\n\nfor param in model.parameters():\n print(param)\n</code></pre>\n<p>Any ideas?</p>",12 "post_number": 1,13 "post_type": 1,14 "posts_count": 4,15 "updated_at": "2020-06-17T21:18:08.678Z",16 "reply_count": 0,17 "reply_to_post_number": null,18 "quote_count": 0,19 "incoming_link_count": 3261,20 "reads": 69,21 "readers_count": 68,22 "score": 16318.8,23 "yours": false,24 "topic_id": 85846,25 "topic_slug": "learning-why-does-unsqueeze-1-magically-work",26 "display_username": "BRETT BYRNES",27 "primary_group_name": null,28 "flair_name": null,29 "flair_url": null,30 "flair_bg_color": null,31 "flair_color": null,32 "flair_group_id": null,33 "badges_granted": [],34 "version": 1,35 "can_edit": false,36 "can_delete": false,37 "can_recover": false,38 "can_see_hidden_post": false,39 "can_wiki": false,40 "read": true,41 "user_title": null,42 "bookmarked": false,43 "actions_summary": [],44 "moderator": false,45 "admin": false,46 "staff": false,47 "user_id": 33199,48 "hidden": false,49 "trust_level": 1,50 "deleted_at": null,51 "user_deleted": false,52 "edit_reason": null,53 "can_view_edit_history": true,54 "wiki": false,55 "post_url": "/t/learning-why-does-unsqueeze-1-magically-work/85846/1",56 "can_accept_answer": false,57 "can_unaccept_answer": false,58 "accepted_answer": false,59 "topic_accepted_answer": null,60 "can_vote": false61 },62 {63 "id": 204077,64 "name": "",65 "username": "harsha_g",66 "avatar_template": "/letter_avatar_proxy/v4/letter/h/4491bb/{size}.png",67 "created_at": "2020-06-17T21:34:16.094Z",68 "cooked": "<p>There’s nothing magic about this. What you got is a warning and not an error. If you look at the documentation for MSELoss it says the target should have the same shape as the input which is what the <code>unsqueeze</code> operation ensures by adding an extra dimension. Hope this helps.</p>",69 "post_number": 2,70 "post_type": 1,71 "posts_count": 4,72 "updated_at": "2020-06-17T21:34:16.094Z",73 "reply_count": 1,74 "reply_to_post_number": null,75 "quote_count": 0,76 "incoming_link_count": 6,77 "reads": 59,78 "readers_count": 58,79 "score": 46.8,80 "yours": false,81 "topic_id": 85846,82 "topic_slug": "learning-why-does-unsqueeze-1-magically-work",83 "display_username": "",84 "primary_group_name": null,85 "flair_name": null,86 "flair_url": null,87 "flair_bg_color": null,88 "flair_color": null,89 "flair_group_id": null,90 "badges_granted": [],91 "version": 1,92 "can_edit": false,93 "can_delete": false,94 "can_recover": false,95 "can_see_hidden_post": false,96 "can_wiki": false,97 "read": true,98 "user_title": null,99 "bookmarked": false,100 "actions_summary": [],101 "moderator": false,102 "admin": false,103 "staff": false,104 "user_id": 32695,105 "hidden": false,106 "trust_level": 2,107 "deleted_at": null,108 "user_deleted": false,109 "edit_reason": null,110 "can_view_edit_history": true,111 "wiki": false,112 "post_url": "/t/learning-why-does-unsqueeze-1-magically-work/85846/2",113 "can_accept_answer": false,114 "can_unaccept_answer": false,115 "accepted_answer": false,116 "topic_accepted_answer": null117 },118 {119 "id": 204085,120 "name": "BRETT BYRNES",121 "username": "btb",122 "avatar_template": "/letter_avatar_proxy/v4/letter/b/9fc348/{size}.png",123 "created_at": "2020-06-18T00:05:29.399Z",124 "cooked": "<p>Ok - so thanks for writing that out, it helps.</p>\n<p>Basically my model is specified output as [y,1] (pred here) so I have to specify the other input to the loss function also as a [y,1] instead of just [y].</p>\n<p>Is that a fair explanation?</p>",125 "post_number": 3,126 "post_type": 1,127 "posts_count": 4,128 "updated_at": "2020-06-18T00:05:29.399Z",129 "reply_count": 1,130 "reply_to_post_number": 2,131 "quote_count": 0,132 "incoming_link_count": 4,133 "reads": 52,134 "readers_count": 51,135 "score": 35.4,136 "yours": false,137 "topic_id": 85846,138 "topic_slug": "learning-why-does-unsqueeze-1-magically-work",139 "display_username": "BRETT BYRNES",140 "primary_group_name": null,141 "flair_name": null,142 "flair_url": null,143 "flair_bg_color": null,144 "flair_color": null,145 "flair_group_id": null,146 "badges_granted": [],147 "version": 1,148 "can_edit": false,149 "can_delete": false,150 "can_recover": false,151 "can_see_hidden_post": false,152 "can_wiki": false,153 "read": true,154 "user_title": null,155 "reply_to_user": {156 "id": 32695,157 "username": "harsha_g",158 "name": "",159 "avatar_template": "/letter_avatar_proxy/v4/letter/h/4491bb/{size}.png"160 },161 "bookmarked": false,162 "actions_summary": [],163 "moderator": false,164 "admin": false,165 "staff": false,166 "user_id": 33199,167 "hidden": false,168 "trust_level": 1,169 "deleted_at": null,170 "user_deleted": false,171 "edit_reason": null,172 "can_view_edit_history": true,173 "wiki": false,174 "post_url": "/t/learning-why-does-unsqueeze-1-magically-work/85846/3",175 "can_accept_answer": false,176 "can_unaccept_answer": false,177 "accepted_answer": false,178 "topic_accepted_answer": null179 },180 {181 "id": 204096,182 "name": "",183 "username": "harsha_g",184 "avatar_template": "/letter_avatar_proxy/v4/letter/h/4491bb/{size}.png",185 "created_at": "2020-06-18T01:23:22.405Z",186 "cooked": "<p>Yes. And vice versa too. If your targets are shaped [y], you shape the output from the model as [y].</p>",187 "post_number": 4,188 "post_type": 1,189 "posts_count": 4,190 "updated_at": "2020-06-18T01:23:22.405Z",191 "reply_count": 0,192 "reply_to_post_number": 3,193 "quote_count": 0,194 "incoming_link_count": 5,195 "reads": 44,196 "readers_count": 43,197 "score": 33.8,198 "yours": false,199 "topic_id": 85846,200 "topic_slug": "learning-why-does-unsqueeze-1-magically-work",201 "display_username": "",202 "primary_group_name": null,203 "flair_name": null,204 "flair_url": null,205 "flair_bg_color": null,206 "flair_color": null,207 "flair_group_id": null,208 "badges_granted": [],209 "version": 1,210 "can_edit": false,211 "can_delete": false,212 "can_recover": false,213 "can_see_hidden_post": false,214 "can_wiki": false,215 "read": true,216 "user_title": null,217 "reply_to_user": {218 "id": 33199,219 "username": "btb",220 "name": "BRETT BYRNES",221 "avatar_template": "/letter_avatar_proxy/v4/letter/b/9fc348/{size}.png"222 },223 "bookmarked": false,224 "actions_summary": [],225 "moderator": false,226 "admin": false,227 "staff": false,228 "user_id": 32695,229 "hidden": false,230 "trust_level": 2,231 "deleted_at": null,232 "user_deleted": false,233 "edit_reason": null,234 "can_view_edit_history": true,235 "wiki": false,236 "post_url": "/t/learning-why-does-unsqueeze-1-magically-work/85846/4",237 "can_accept_answer": false,238 "can_unaccept_answer": false,239 "accepted_answer": false,240 "topic_accepted_answer": null241 }242 ],243 "stream": [244 204075,245 204077,246 204085,247 204096248 ]249 },250 "timeline_lookup": [251 [252 1,253 1956254 ]255 ],256 "suggested_topics": [257 {258 "fancy_title": "GradScaler: TypeError: Cannot convert a MPS Tensor to float64 dtype as the MPS framework doesn’t support float64. Please use float32 instead",259 "id": 213206,260 "title": "GradScaler: TypeError: Cannot convert a MPS Tensor to float64 dtype as the MPS framework doesn't support float64. Please use float32 instead",261 "slug": "gradscaler-typeerror-cannot-convert-a-mps-tensor-to-float64-dtype-as-the-mps-framework-doesnt-support-float64-please-use-float32-instead",262 "posts_count": 2,263 "reply_count": 0,264 "highest_post_number": 2,265 "image_url": null,266 "created_at": "2024-11-20T12:09:55.629Z",267 "last_posted_at": "2024-12-23T15:29:26.138Z",268 "bumped": true,269 "bumped_at": "2024-12-23T15:29:26.138Z",270 "archetype": "regular",271 "unseen": false,272 "pinned": false,273 "unpinned": null,274 "visible": true,275 "closed": false,276 "archived": false,277 "bookmarked": null,278 "liked": null,279 "tags_descriptions": {},280 "like_count": 0,281 "views": 658,282 "category_id": 1,283 "featured_link": null,284 "has_accepted_answer": false,285 "posters": [286 {287 "extras": null,288 "description": "Original Poster",289 "user": {290 "id": 80423,291 "username": "Mauro_Sciancalepore",292 "name": "Mauro Sciancalepore",293 "avatar_template": "/user_avatar/discuss.pytorch.org/mauro_sciancalepore/{size}/73510_2.png",294 "trust_level": 1295 }296 },297 {298 "extras": "latest",299 "description": "Most Recent Poster",300 "user": {301 "id": 81681,302 "username": "starksm64",303 "name": "Scott M Stark",304 "avatar_template": "/user_avatar/discuss.pytorch.org/starksm64/{size}/74694_2.png",305 "trust_level": 0306 }307 }308 ]309 },310 {311 "fancy_title": "Find maximum length of consecutive zeros in each row",312 "id": 213871,313 "title": "Find maximum length of consecutive zeros in each row",314 "slug": "find-maximum-length-of-consecutive-zeros-in-each-row",315 "posts_count": 3,316 "reply_count": 0,317 "highest_post_number": 3,318 "image_url": null,319 "created_at": "2024-12-05T16:46:38.153Z",320 "last_posted_at": "2024-12-08T19:58:20.444Z",321 "bumped": true,322 "bumped_at": "2024-12-08T19:58:20.444Z",323 "archetype": "regular",324 "unseen": false,325 "pinned": false,326 "unpinned": null,327 "visible": true,328 "closed": false,329 "archived": false,330 "bookmarked": null,331 "liked": null,332 "tags_descriptions": {},333 "like_count": 0,334 "views": 215,335 "category_id": 1,336 "featured_link": null,337 "has_accepted_answer": false,338 "posters": [339 {340 "extras": null,341 "description": "Original Poster",342 "user": {343 "id": 81339,344 "username": "Paulo_Nascimento",345 "name": "Paulo Nascimento",346 "avatar_template": "/user_avatar/discuss.pytorch.org/paulo_nascimento/{size}/72888_2.png",347 "trust_level": 0348 }349 },350 {351 "extras": null,352 "description": "Frequent Poster",353 "user": {354 "id": 72430,355 "username": "Eduardo_Lawson",356 "name": "Eduardo Lawson da Silva",357 "avatar_template": "/user_avatar/discuss.pytorch.org/eduardo_lawson/{size}/66899_2.png",358 "trust_level": 2359 }360 },361 {362 "extras": "latest",363 "description": "Most Recent Poster",364 "user": {365 "id": 18088,366 "username": "KFrank",367 "name": "K. Frank",368 "avatar_template": "/letter_avatar_proxy/v4/letter/k/ecb155/{size}.png",369 "trust_level": 2370 }371 }372 ]373 },374 {375 "fancy_title": "How to (efficiently) apply a function without a “dim” argument to each row of a 2D tensor?",376 "id": 215365,377 "title": "How to (efficiently) apply a function without a \"dim\" argument to each row of a 2D tensor?",378 "slug": "how-to-efficiently-apply-a-function-without-a-dim-argument-to-each-row-of-a-2d-tensor",379 "posts_count": 1,380 "reply_count": 0,381 "highest_post_number": 1,382 "image_url": null,383 "created_at": "2025-01-14T10:42:36.374Z",384 "last_posted_at": "2025-01-14T10:42:36.448Z",385 "bumped": true,386 "bumped_at": "2025-01-14T10:42:36.448Z",387 "archetype": "regular",388 "unseen": false,389 "pinned": false,390 "unpinned": null,391 "visible": true,392 "closed": false,393 "archived": false,394 "bookmarked": null,395 "liked": null,396 "tags_descriptions": {},397 "like_count": 0,398 "views": 83,399 "category_id": 1,400 "featured_link": null,401 "has_accepted_answer": false,402 "posters": [403 {404 "extras": "latest single",405 "description": "Original Poster, Most Recent Poster",406 "user": {407 "id": 82080,408 "username": "Matt_T1",409 "name": "Matt T.",410 "avatar_template": "/user_avatar/discuss.pytorch.org/matt_t1/{size}/75097_2.png",411 "trust_level": 1412 }413 }414 ]415 },416 {417 "fancy_title": "Understanding the traceback of a compiler error with cache size limit",418 "id": 215669,419 "title": "Understanding the traceback of a compiler error with cache size limit",420 "slug": "understanding-the-traceback-of-a-compiler-error-with-cache-size-limit",421 "posts_count": 1,422 "reply_count": 0,423 "highest_post_number": 1,424 "image_url": null,425 "created_at": "2025-01-21T10:43:52.773Z",426 "last_posted_at": "2025-01-21T10:43:52.812Z",427 "bumped": true,428 "bumped_at": "2025-01-21T10:43:52.812Z",429 "archetype": "regular",430 "unseen": false,431 "pinned": false,432 "unpinned": null,433 "visible": true,434 "closed": false,435 "archived": false,436 "bookmarked": null,437 "liked": null,438 "tags_descriptions": {},439 "like_count": 0,440 "views": 53,441 "category_id": 1,442 "featured_link": null,443 "has_accepted_answer": false,444 "posters": [445 {446 "extras": "latest single",447 "description": "Original Poster, Most Recent Poster",448 "user": {449 "id": 82217,450 "username": "Johannes_Vogt",451 "name": "Johannes Vogt",452 "avatar_template": "/user_avatar/discuss.pytorch.org/johannes_vogt/{size}/75220_2.png",453 "trust_level": 1454 }455 }456 ]457 },458 {459 "fancy_title": "Call order of hooks",460 "id": 215863,461 "title": "Call order of hooks",462 "slug": "call-order-of-hooks",463 "posts_count": 1,464 "reply_count": 0,465 "highest_post_number": 1,466 "image_url": null,467 "created_at": "2025-01-25T16:57:33.402Z",468 "last_posted_at": "2025-01-25T16:57:33.447Z",469 "bumped": true,470 "bumped_at": "2025-01-25T16:57:33.447Z",471 "archetype": "regular",472 "unseen": false,473 "pinned": false,474 "unpinned": null,475 "visible": true,476 "closed": false,477 "archived": false,478 "bookmarked": null,479 "liked": null,480 "tags_descriptions": {},481 "like_count": 0,482 "views": 70,483 "category_id": 1,484 "featured_link": null,485 "has_accepted_answer": false,486 "posters": [487 {488 "extras": "latest single",489 "description": "Original Poster, Most Recent Poster",490 "user": {491 "id": 82245,492 "username": "sternj",493 "name": null,494 "avatar_template": "/letter_avatar_proxy/v4/letter/s/b19c9b/{size}.png",495 "trust_level": 1496 }497 }498 ]499 }500 ],501 "tags_descriptions": {},502 "fancy_title": "Learning - Why does .unsqueeze(-1) magically work?",503 "id": 85846,504 "title": "Learning - Why does .unsqueeze(-1) magically work?",505 "posts_count": 4,506 "created_at": "2020-06-17T21:18:08.608Z",507 "views": 4648,508 "reply_count": 2,509 "like_count": 0,510 "last_posted_at": "2020-06-18T01:23:22.405Z",511 "visible": true,512 "closed": false,513 "archived": false,514 "has_summary": false,515 "archetype": "regular",516 "slug": "learning-why-does-unsqueeze-1-magically-work",517 "category_id": 1,518 "word_count": 490,519 "deleted_at": null,520 "user_id": 33199,521 "featured_link": null,522 "pinned_globally": false,523 "pinned_at": null,524 "pinned_until": null,525 "image_url": null,526 "slow_mode_seconds": 0,527 "draft": null,528 "draft_key": "topic_85846",529 "draft_sequence": null,530 "unpinned": null,531 "pinned": false,532 "current_post_number": 1,533 "highest_post_number": 4,534 "deleted_by": null,535 "actions_summary": [536 {537 "id": 4,538 "count": 0,539 "hidden": false,540 "can_act": false541 },542 {543 "id": 8,544 "count": 0,545 "hidden": false,546 "can_act": false547 },548 {549 "id": 10,550 "count": 0,551 "hidden": false,552 "can_act": false553 },554 {555 "id": 7,556 "count": 0,557 "hidden": false,558 "can_act": false559 }560 ],561 "chunk_size": 20,562 "bookmarked": false,563 "topic_timer": null,564 "message_bus_last_id": 0,565 "participant_count": 2,566 "show_read_indicator": false,567 "thumbnails": null,568 "slow_mode_enabled_until": null,569 "can_vote": false,570 "vote_count": 0,571 "user_voted": false,572 "discourse_zendesk_plugin_zendesk_id": null,573 "discourse_zendesk_plugin_zendesk_url": "https://your-url.zendesk.com/agent/tickets/",574 "details": {575 "can_edit": false,576 "notification_level": 1,577 "participants": [578 {579 "id": 32695,580 "username": "harsha_g",581 "name": "",582 "avatar_template": "/letter_avatar_proxy/v4/letter/h/4491bb/{size}.png",583 "post_count": 2,584 "primary_group_name": null,585 "flair_name": null,586 "flair_url": null,587 "flair_color": null,588 "flair_bg_color": null,589 "flair_group_id": null,590 "trust_level": 2591 },592 {593 "id": 33199,594 "username": "btb",595 "name": "BRETT BYRNES",596 "avatar_template": "/letter_avatar_proxy/v4/letter/b/9fc348/{size}.png",597 "post_count": 2,598 "primary_group_name": null,599 "flair_name": null,600 "flair_url": null,601 "flair_color": null,602 "flair_bg_color": null,603 "flair_group_id": null,604 "trust_level": 1605 }606 ],607 "created_by": {608 "id": 33199,609 "username": "btb",610 "name": "BRETT BYRNES",611 "avatar_template": "/letter_avatar_proxy/v4/letter/b/9fc348/{size}.png"612 },613 "last_poster": {614 "id": 32695,615 "username": "harsha_g",616 "name": "",617 "avatar_template": "/letter_avatar_proxy/v4/letter/h/4491bb/{size}.png"618 }619 },620 "bookmarks": []621 },622 {623 "post_stream": {624 "posts": [625 {626 "id": 204087,627 "name": "André Teixeira",628 "username": "AndreTeixeira",629 "avatar_template": "/user_avatar/discuss.pytorch.org/andreteixeira/{size}/22945_2.png",630 "created_at": "2020-06-18T00:31:04.051Z",631 "cooked": "<p>I’m training a CNN for plant diseases recognition and the training set is composed by laboratory images. I am achieving low precision in my validation and test sets as they are composed of images taken in the field and which contain an extremely varied dynamic range. Is there a way to introduce data augmentation on my training set based on the validation set in order to get a better acurracy?</p>",632 "post_number": 1,633 "post_type": 1,634 "posts_count": 2,635 "updated_at": "2020-06-18T00:31:21.651Z",636 "reply_count": 0,637 "reply_to_post_number": null,638 "quote_count": 0,639 "incoming_link_count": 31,640 "reads": 6,641 "readers_count": 5,642 "score": 156.2,643 "yours": false,644 "topic_id": 85852,645 "topic_slug": "is-there-a-way-to-insert-data-augmentation-into-the-training-data-based-on-the-validation-data",646 "display_username": "André Teixeira",647 "primary_group_name": null,648 "flair_name": null,649 "flair_url": null,650 "flair_bg_color": null,651 "flair_color": null,652 "flair_group_id": null,653 "badges_granted": [],654 "version": 1,655 "can_edit": false,656 "can_delete": false,657 "can_recover": false,658 "can_see_hidden_post": false,659 "can_wiki": false,660 "read": true,661 "user_title": null,662 "bookmarked": false,663 "actions_summary": [],664 "moderator": false,665 "admin": false,666 "staff": false,667 "user_id": 30206,668 "hidden": false,669 "trust_level": 1,670 "deleted_at": null,671 "user_deleted": false,672 "edit_reason": null,673 "can_view_edit_history": true,674 "wiki": false,675 "post_url": "/t/is-there-a-way-to-insert-data-augmentation-into-the-training-data-based-on-the-validation-data/85852/1",676 "can_accept_answer": false,677 "can_unaccept_answer": false,678 "accepted_answer": false,679 "topic_accepted_answer": null,680 "can_vote": false681 },682 {683 "id": 204088,684 "name": "Nikan Doosti",685 "username": "Nikronic",686 "avatar_template": "/user_avatar/discuss.pytorch.org/nikronic/{size}/71873_2.png",687 "created_at": "2020-06-18T00:40:47.579Z",688 "cooked": "<p>Hi,</p>\n<p>I am not sure I have understood your question properly, but I think the problem is for instance, train set has normal images, but val set has rotated images or many other issues that differs from train set and you want to introduce some augmentations like rotation to make training set more similar to val.</p>\n<p>If it is that case, I have to mention a few points:</p>\n<ol>\n<li>Introducing any new statistical changes to training based on validation set is some kind of cheating as in real world, we know that we are going to test our algorithm against samples that it has never seen. So actually, you have seen val set, tested your model got output and now you know model is not working well. So, I think your idea is not really acceptable to change train set to be similar to val set.</li>\n<li>Why not shuffling lab and field images? create a new train set that has samples in val set.</li>\n<li>I am not sure about this but how about adding all type of augmentations to make sure all possible transformation can be learned by network.</li>\n<li>I thing there was a paper called <em>auto augment</em> which tried to find best augmentation based on data.</li>\n</ol>\n<p>Bests</p>",689 "post_number": 2,690 "post_type": 1,691 "posts_count": 2,692 "updated_at": "2020-06-18T00:40:47.579Z",693 "reply_count": 0,694 "reply_to_post_number": null,695 "quote_count": 0,696 "incoming_link_count": 0,697 "reads": 6,698 "readers_count": 5,699 "score": 1.2,700 "yours": false,701 "topic_id": 85852,702 "topic_slug": "is-there-a-way-to-insert-data-augmentation-into-the-training-data-based-on-the-validation-data",703 "display_username": "Nikan Doosti",704 "primary_group_name": null,705 "flair_name": null,706 "flair_url": null,707 "flair_bg_color": null,708 "flair_color": null,709 "flair_group_id": null,710 "badges_granted": [],711 "version": 1,712 "can_edit": false,713 "can_delete": false,714 "can_recover": false,715 "can_see_hidden_post": false,716 "can_wiki": false,717 "read": true,718 "user_title": "",719 "bookmarked": false,720 "actions_summary": [],721 "moderator": false,722 "admin": false,723 "staff": false,724 "user_id": 12783,725 "hidden": false,726 "trust_level": 2,727 "deleted_at": null,728 "user_deleted": false,729 "edit_reason": null,730 "can_view_edit_history": true,731 "wiki": false,732 "post_url": "/t/is-there-a-way-to-insert-data-augmentation-into-the-training-data-based-on-the-validation-data/85852/2",733 "can_accept_answer": false,734 "can_unaccept_answer": false,735 "accepted_answer": false,736 "topic_accepted_answer": null737 }738 ],739 "stream": [740 204087,741 204088742 ]743 },744 "timeline_lookup": [745 [746 1,747 1956748 ]749 ],750 "suggested_topics": [751 {752 "fancy_title": "Persistent error in loss.backward() during joint-training of two models using two different but dependent loss functions",753 "id": 212593,754 "title": "Persistent error in loss.backward() during joint-training of two models using two different but dependent loss functions",755 "slug": "persistent-error-in-loss-backward-during-joint-training-of-two-models-using-two-different-but-dependent-loss-functions",756 "posts_count": 1,757 "reply_count": 0,758 "highest_post_number": 1,759 "image_url": "https://discuss.pytorch.org/uploads/default/optimized/3X/3/d/3db4085e6be45f34ad9d0a6ff92d9f7a375fa71d_2_1024x664.png",760 "created_at": "2024-11-06T05:47:29.578Z",761 "last_posted_at": "2024-11-06T05:47:29.688Z",762 "bumped": true,763 "bumped_at": "2024-11-06T05:47:29.688Z",764 "archetype": "regular",765 "unseen": false,766 "pinned": false,767 "unpinned": null,768 "visible": true,769 "closed": false,770 "archived": false,771 "bookmarked": null,772 "liked": null,773 "tags_descriptions": {},774 "like_count": 0,775 "views": 10,776 "category_id": 1,777 "featured_link": null,778 "has_accepted_answer": false,779 "posters": [780 {781 "extras": "latest single",782 "description": "Original Poster, Most Recent Poster",783 "user": {784 "id": 80719,785 "username": "Goutam",786 "name": "Goutam",787 "avatar_template": "/user_avatar/discuss.pytorch.org/goutam/{size}/73797_2.png",788 "trust_level": 0789 }790 }791 ]792 },793 {794 "fancy_title": "GradCAM while training break gradients flow",795 "id": 215992,796 "title": "GradCAM while training break gradients flow",797 "slug": "gradcam-while-training-break-gradients-flow",798 "posts_count": 1,799 "reply_count": 0,800 "highest_post_number": 1,801 "image_url": null,802 "created_at": "2025-01-28T14:52:46.641Z",803 "last_posted_at": "2025-01-28T14:52:46.685Z",804 "bumped": true,805 "bumped_at": "2025-01-28T14:52:46.685Z",806 "archetype": "regular",807 "unseen": false,808 "pinned": false,809 "unpinned": null,810 "visible": true,811 "closed": false,812 "archived": false,813 "bookmarked": null,814 "liked": null,815 "tags_descriptions": {},816 "like_count": 0,817 "views": 113,818 "category_id": 1,819 "featured_link": null,820 "has_accepted_answer": false,821 "posters": [822 {823 "extras": "latest single",824 "description": "Original Poster, Most Recent Poster",825 "user": {826 "id": 34558,827 "username": "Giovanni_Cena",828 "name": "Giovanni Cena",829 "avatar_template": "/user_avatar/discuss.pytorch.org/giovanni_cena/{size}/16672_2.png",830 "trust_level": 1831 }832 }833 ]834 },835 {836 "fancy_title": "Incorporate Moving Average Into Loss",837 "id": 216910,838 "title": "Incorporate Moving Average Into Loss",839 "slug": "incorporate-moving-average-into-loss",840 "posts_count": 1,841 "reply_count": 0,842 "highest_post_number": 1,843 "image_url": null,844 "created_at": "2025-02-19T20:47:42.880Z",845 "last_posted_at": "2025-02-19T20:47:42.924Z",846 "bumped": true,847 "bumped_at": "2025-02-20T06:34:35.978Z",848 "archetype": "regular",849 "unseen": false,850 "pinned": false,851 "unpinned": null,852 "visible": true,853 "closed": false,854 "archived": false,855 "bookmarked": null,856 "liked": null,857 "tags_descriptions": {},858 "like_count": 0,859 "views": 63,860 "category_id": 1,861 "featured_link": null,862 "has_accepted_answer": false,863 "posters": [864 {865 "extras": "latest single",866 "description": "Original Poster, Most Recent Poster",867 "user": {868 "id": 56635,869 "username": "jean_b",870 "name": "",871 "avatar_template": "/letter_avatar_proxy/v4/letter/j/7c8e57/{size}.png",872 "trust_level": 1873 }874 }875 ]876 },877 {878 "fancy_title": "Permute inside nn.Sequential",879 "id": 214877,880 "title": "Permute inside nn.Sequential",881 "slug": "permute-inside-nn-sequential",882 "posts_count": 3,883 "reply_count": 0,884 "highest_post_number": 3,885 "image_url": null,886 "created_at": "2025-01-02T08:19:47.254Z",887 "last_posted_at": "2025-09-15T19:19:55.247Z",888 "bumped": true,889 "bumped_at": "2025-09-15T19:19:55.247Z",890 "archetype": "regular",891 "unseen": false,892 "pinned": false,893 "unpinned": null,894 "visible": true,895 "closed": false,896 "archived": false,897 "bookmarked": null,898 "liked": null,899 "tags_descriptions": {},900 "like_count": 1,901 "views": 476,902 "category_id": 1,903 "featured_link": null,904 "has_accepted_answer": false,905 "posters": [906 {907 "extras": null,908 "description": "Original Poster",909 "user": {910 "id": 81597,911 "username": "nada",912 "name": null,913 "avatar_template": "/letter_avatar_proxy/v4/letter/n/a88e4f/{size}.png",914 "trust_level": 1915 }916 },917 {918 "extras": null,919 "description": "Frequent Poster",920 "user": {921 "id": 3534,922 "username": "ptrblck",923 "name": "",924 "avatar_template": "/user_avatar/discuss.pytorch.org/ptrblck/{size}/1823_2.png",925 "admin": true,926 "moderator": true,927 "trust_level": 2928 }929 },930 {931 "extras": "latest",932 "description": "Most Recent Poster",933 "user": {934 "id": 32812,935 "username": "Bjorn_Lindqvist",936 "name": "Björn Lindqvist",937 "avatar_template": "/user_avatar/discuss.pytorch.org/bjorn_lindqvist/{size}/25326_2.png",938 "trust_level": 2939 }940 }941 ]942 },943 {944 "fancy_title": "How do I remove torch_dtype=torch.float16 on Windows 11?",945 "id": 218446,946 "title": "How do I remove torch_dtype=torch.float16 on Windows 11?",947 "slug": "how-do-i-remove-torch-dtype-torch-float16-on-windows-11",948 "posts_count": 3,949 "reply_count": 1,950 "highest_post_number": 3,951 "image_url": null,952 "created_at": "2025-03-31T13:34:36.413Z",953 "last_posted_at": "2025-04-15T18:54:42.340Z",954 "bumped": true,955 "bumped_at": "2025-04-15T18:54:42.340Z",956 "archetype": "regular",957 "unseen": false,958 "pinned": false,959 "unpinned": null,960 "visible": true,961 "closed": false,962 "archived": false,963 "bookmarked": null,964 "liked": null,965 "tags_descriptions": {},966 "like_count": 0,967 "views": 182,968 "category_id": 1,969 "featured_link": null,970 "has_accepted_answer": false,971 "posters": [972 {973 "extras": null,974 "description": "Original Poster",975 "user": {976 "id": 83545,977 "username": "Taurus1983",978 "name": "",979 "avatar_template": "/letter_avatar_proxy/v4/letter/t/e9bcb4/{size}.png",980 "trust_level": 0981 }982 },983 {984 "extras": null,985 "description": "Frequent Poster",986 "user": {987 "id": 3534,988 "username": "ptrblck",989 "name": "",990 "avatar_template": "/user_avatar/discuss.pytorch.org/ptrblck/{size}/1823_2.png",991 "admin": true,992 "moderator": true,993 "trust_level": 2994 }995 },996 {997 "extras": "latest",998 "description": "Most Recent Poster",999 "user": {1000 "id": 83836,1001 "username": "rxrue",1002 "name": "",1003 "avatar_template": "/user_avatar/discuss.pytorch.org/rxrue/{size}/76643_2.png",1004 "trust_level": 01005 }1006 }1007 ]1008 }1009 ],1010 "tags_descriptions": {},1011 "fancy_title": "Is there a way to insert data augmentation into the training data based on the validation data?",1012 "id": 85852,1013 "title": "Is there a way to insert data augmentation into the training data based on the validation data?",1014 "posts_count": 2,1015 "created_at": "2020-06-18T00:31:03.990Z",1016 "views": 288,1017 "reply_count": 0,1018 "like_count": 0,1019 "last_posted_at": "2020-06-18T00:40:47.579Z",1020 "visible": true,1021 "closed": false,1022 "archived": false,1023 "has_summary": false,1024 "archetype": "regular",1025 "slug": "is-there-a-way-to-insert-data-augmentation-into-the-training-data-based-on-the-validation-data",1026 "category_id": 1,1027 "word_count": 282,1028 "deleted_at": null,1029 "user_id": 30206,1030 "featured_link": null,1031 "pinned_globally": false,1032 "pinned_at": null,1033 "pinned_until": null,1034 "image_url": null,1035 "slow_mode_seconds": 0,1036 "draft": null,1037 "draft_key": "topic_85852",1038 "draft_sequence": null,1039 "unpinned": null,1040 "pinned": false,1041 "current_post_number": 1,1042 "highest_post_number": 2,1043 "deleted_by": null,1044 "actions_summary": [1045 {1046 "id": 4,1047 "count": 0,1048 "hidden": false,1049 "can_act": false1050 },1051 {1052 "id": 8,1053 "count": 0,1054 "hidden": false,1055 "can_act": false1056 },1057 {1058 "id": 10,1059 "count": 0,1060 "hidden": false,1061 "can_act": false1062 },1063 {1064 "id": 7,1065 "count": 0,1066 "hidden": false,1067 "can_act": false1068 }1069 ],1070 "chunk_size": 20,1071 "bookmarked": false,1072 "topic_timer": null,1073 "message_bus_last_id": 0,1074 "participant_count": 2,1075 "show_read_indicator": false,1076 "thumbnails": null,1077 "slow_mode_enabled_until": null,1078 "can_vote": false,1079 "vote_count": 0,1080 "user_voted": false,1081 "discourse_zendesk_plugin_zendesk_id": null,1082 "discourse_zendesk_plugin_zendesk_url": "https://your-url.zendesk.com/agent/tickets/",1083 "details": {1084 "can_edit": false,1085 "notification_level": 1,1086 "participants": [1087 {1088 "id": 12783,1089 "username": "Nikronic",1090 "name": "Nikan Doosti",1091 "avatar_template": "/user_avatar/discuss.pytorch.org/nikronic/{size}/71873_2.png",1092 "post_count": 1,1093 "primary_group_name": null,1094 "flair_name": null,1095 "flair_url": null,1096 "flair_color": null,1097 "flair_bg_color": null,1098 "flair_group_id": null,1099 "trust_level": 21100 },1101 {1102 "id": 30206,1103 "username": "AndreTeixeira",1104 "name": "André Teixeira",1105 "avatar_template": "/user_avatar/discuss.pytorch.org/andreteixeira/{size}/22945_2.png",1106 "post_count": 1,1107 "primary_group_name": null,1108 "flair_name": null,1109 "flair_url": null,1110 "flair_color": null,1111 "flair_bg_color": null,1112 "flair_group_id": null,1113 "trust_level": 11114 }1115 ],1116 "created_by": {1117 "id": 30206,1118 "username": "AndreTeixeira",1119 "name": "André Teixeira",1120 "avatar_template": "/user_avatar/discuss.pytorch.org/andreteixeira/{size}/22945_2.png"1121 },1122 "last_poster": {1123 "id": 12783,1124 "username": "Nikronic",1125 "name": "Nikan Doosti",1126 "avatar_template": "/user_avatar/discuss.pytorch.org/nikronic/{size}/71873_2.png"1127 }1128 },1129 "bookmarks": []1130 },1131 {1132 "post_stream": {1133 "posts": [1134 {1135 "id": 12060,1136 "name": "Andrew Kirillov",1137 "username": "AndrewKirillo",1138 "avatar_template": "/letter_avatar_proxy/v4/letter/a/ad7895/{size}.png",1139 "created_at": "2017-07-10T19:16:34.758Z",1140 "cooked": "<p>Hey guys.<br>\nSo I’m very new to PyTorch and Neural Networks in general, and I’m having some problems creating a Neural Network that classifies names by gender.<br>\nI based this off of the PyTorch tutorial for RNNs that classify names by nationality, but I decided not to go with a recurrent approach… <em>Stop me right here if this was the wrong idea!</em><br>\nHowever, whenever I try to run an input through the network it tells me:<br>\n<code>RuntimeError: matrices expected, got 3D, 2D tensors at /py/conda-bld/pytorch_1493681908901/work/torch/lib/TH/generic/THTensorMath.c:1232</code><br>\nI know this has something to do with how PyTorch always expects there to be a batch size or something, and I have my tensor set up that way, but you can probably tell by this point that I have no idea what I’m talking about.<br>\nHere’s my code:<br>\nfrom <strong>future</strong> import unicode_literals, print_function, division<br>\nfrom io import open<br>\nimport glob<br>\nimport unicodedata<br>\nimport string<br>\nimport torch<br>\nimport torchvision<br>\nimport torch.nn as nn<br>\nimport torch.optim as optim<br>\nimport random<br>\nfrom torch.autograd import Variable<br>\nimport matplotlib.pyplot as plt<br>\nimport matplotlib.ticker as ticker</p>\n<pre><code>\"\"\"------GLOBAL VARIABLES------\"\"\"\n\nall_letters = string.ascii_letters + \" .,;'\"\nnum_letters = len(all_letters)\nall_names = {}\ngenders = [\"Female\", \"Male\"]\n\n\"\"\"-------DATA EXTRACTION------\"\"\"\n\ndef findFiles(path):\n return glob.glob(path)\n\ndef unicodeToAscii(s):\n return ''.join(\n c for c in unicodedata.normalize('NFD', s)\n if unicodedata.category(c) != 'Mn'\n and c in all_letters\n )\n\n# Read a file and split into lines\ndef readLines(filename):\n lines = open(filename, encoding='utf-8').read().strip().split('\\n')\n return [unicodeToAscii(line) for line in lines]\n\nfor file in findFiles(\"/home/andrew/PyCharm/PycharmProjects/CantStop/data/names/*.txt\"):\n gender = file.split(\"/\")[-1].split(\".\")[0]\n names = readLines(file)\n all_names[gender] = names\n\n\"\"\"-----DATA INTERPRETATION-----\"\"\"\n\ndef nameToTensor(name):\n tensor = torch.zeros(len(name), 1, num_letters)\n for index, letter in enumerate(name):\n tensor[index][0][all_letters.find(letter)] = 1\n return tensor\n\ndef outputToGender(output):\n gender, gender_index = output.data.topk(1)\n if gender_index[0][0] == 0:\n return \"Female\"\n return \"Male\"\n\n\"\"\"------NETWORK SETUP------\"\"\"\n\nclass Net(nn.Module):\n def __init__(self, input_size, output_size):\n super(Net, self).__init__()\n #Layer 1\n self.Lin1 = nn.Linear(input_size, int(input_size/2))\n self.ReLu1 = nn.ReLU()\n self.Batch1 = nn.BatchNorm1d(int(input_size/2))\n #Layer 2\n self.Lin2 = nn.Linear(int(input_size/2), output_size)\n self.ReLu2 = nn.ReLU()\n self.Batch2 = nn.BatchNorm1d(output_size)\n self.softMax = nn.LogSoftmax()\n\n def forward(self, input):\n output1 = self.Batch1(self.ReLu1(self.Lin1(input)))\n output2 = self.softMax(self.Batch2(self.ReLu2(self.Lin2(output1))))\n return output2\n\nNN = Net(num_letters, 2)\n\n\"\"\"------TRAINING------\"\"\"\n\ndef getRandomTrainingEx():\n gender = genders[random.randint(0, 1)]\n name = all_names[gender][random.randint(0, len(all_names[gender])-1)]\n gender_tensor = Variable(torch.LongTensor([genders.index(gender)]))\n name_tensor = Variable(nameToTensor(name))\n return gender_tensor, name_tensor, gender\n\ndef train(input, target):\n loss_func = nn.NLLLoss()\n\n optimizer = optim.SGD(NN.parameters(), lr=0.0001, momentum=0.9)\n\n optimizer.zero_grad()\n\n output = NN(input)\n\n loss = loss_func(output, target)\n loss.backward()\n optimizer.step()\n\n return output, loss\n\nall_losses = []\ncurrent_loss = 0\n\nfor i in range(100000):\n gender_tensor, name_tensor, gender = getRandomTrainingEx()\n output, loss = train(name_tensor, gender_tensor)\n current_loss += loss\n\n if i%1000 == 0:\n print(\"Guess: %s, Correct: %s, Loss: %s\" % (outputToGender(output), gender, loss.data[0]))\n\n if i%100 == 0:\n all_losses.append(current_loss/10)\n current_loss = 0\n\n# plt.figure()\n# plt.plot(all_losses)\n# plt.show()\n</code></pre>\n<p>Please help a newbie out!</p>",1141 "post_number": 1,1142 "post_type": 1,1143 "posts_count": 7,1144 "updated_at": "2017-07-10T19:16:34.758Z",1145 "reply_count": 0,1146 "reply_to_post_number": null,1147 "quote_count": 0,1148 "incoming_link_count": 42673,1149 "reads": 1053,1150 "readers_count": 1052,1151 "score": 213435.6,1152 "yours": false,1153 "topic_id": 4761,1154 "topic_slug": "confused-about-tensor-dimensions-and-batches",1155 "display_username": "Andrew Kirillov",1156 "primary_group_name": null,1157 "flair_name": null,1158 "flair_url": null,1159 "flair_bg_color": null,1160 "flair_color": null,1161 "flair_group_id": null,1162 "badges_granted": [],1163 "version": 1,1164 "can_edit": false,1165 "can_delete": false,1166 "can_recover": false,1167 "can_see_hidden_post": false,1168 "can_wiki": false,1169 "read": true,1170 "user_title": null,1171 "bookmarked": false,1172 "actions_summary": [],1173 "moderator": false,1174 "admin": false,1175 "staff": false,1176 "user_id": 2231,1177 "hidden": false,1178 "trust_level": 1,1179 "deleted_at": null,1180 "user_deleted": false,1181 "edit_reason": null,1182 "can_view_edit_history": true,1183 "wiki": false,1184 "post_url": "/t/confused-about-tensor-dimensions-and-batches/4761/1",1185 "can_accept_answer": false,1186 "can_unaccept_answer": false,1187 "accepted_answer": false,1188 "topic_accepted_answer": null,1189 "can_vote": false1190 },1191 {1192 "id": 12068,1193 "name": "Jordan Campbell",1194 "username": "Jordan_Campbell",1195 "avatar_template": "/user_avatar/discuss.pytorch.org/jordan_campbell/{size}/91_2.png",1196 "created_at": "2017-07-10T21:57:04.441Z",1197 "cooked": "<p>Hi,</p>\n<p>The input to a linear layer should be a tensor of size <code>[batch_size, input_size]</code> where <code>input_size</code> is the same size as the first layer in your network (so in your case it’s <code>num_letters</code>).</p>\n<p>The problem appears in the line:</p>\n<pre><code>tensor = torch.zeros(len(name), 1, num_letters)\n</code></pre>\n<p>which should actually just be:</p>\n<pre><code>tensor = torch.zeros(len(name), num_letters)\n</code></pre>\n<p>As an easy example:</p>\n<pre><code>input_size = 8\noutput_size = 14\nbatch_size = 64\n\nnet = nn.Linear(input_size, output_size)\ninput = Variable(torch.FloatTensor(batch_size, input_size))\n\noutput = net(input)\n\nprint(\"Output size:\", output.size())\n</code></pre>\n<blockquote>\n<p>Output size: (64, 14)</p>\n</blockquote>\n<p>Hope this helps,<br>\nJordan</p>",1198 "post_number": 2,1199 "post_type": 1,1200 "posts_count": 7,