Anurag1734/cuda-error-resolution-analysis
07
1[2 {3 "post_stream": {4 "posts": [5 {6 "id": 223039,7 "name": "",8 "username": "themoonboy",9 "avatar_template": "/user_avatar/discuss.pytorch.org/themoonboy/{size}/13133_2.png",10 "created_at": "2020-08-22T00:19:45.919Z",11 "cooked": "<p>Hi all, I want to ask about the experience to pick up a suitable loss function for unsupervised problems. Here is a description of my current problem.</p>\n<p>Assume I divide the samples into positive and negative, then get two groups of scores when passing them into my model, the dummy codes like:</p>\n<pre><code class=\"lang-auto\">pos_scores = model(pos_samples) # (sample_num, 1)\nneg_socres = model(neg_samples) # (sample_num, 1)\npos_x = pos_scores.mean()\nneg_x = neg_scores.mean()\nloss = LossFunction(pos_x, neg_x)\n</code></pre>\n<p>Where I want to <strong>maximize the pos_scores while minimizing the neg_scores at the same time</strong>, there is no limitation of their values. Note that it is an unsupervised problem, thus no ground-truth label as a reference, only a calculated score for each sample.</p>\n<p>One loss function I have tried is:</p>\n<p><code> loss = neg_x - pos_x</code></p>\n<p>by minimizing this loss function, the code tends to minimize neg_x while maximizing pos_x. But the resulted scores are not very brilliant. Thus, are there any other useful functions that could help to handle with this problem? Thanks all!</p>",12 "post_number": 1,13 "post_type": 1,14 "posts_count": 1,15 "updated_at": "2020-08-22T00:22:00.924Z",16 "reply_count": 0,17 "reply_to_post_number": null,18 "quote_count": 0,19 "incoming_link_count": 13,20 "reads": 3,21 "readers_count": 2,22 "score": 65.6,23 "yours": false,24 "topic_id": 93684,25 "topic_slug": "pickup-a-loss-function-for-maximizing-pos-x-while-minimizing-neg-x",26 "display_username": "",27 "primary_group_name": null,28 "flair_name": null,29 "flair_url": null,30 "flair_bg_color": null,31 "flair_color": null,32 "flair_group_id": null,33 "badges_granted": [],34 "version": 1,35 "can_edit": false,36 "can_delete": false,37 "can_recover": false,38 "can_see_hidden_post": false,39 "can_wiki": false,40 "read": true,41 "user_title": null,42 "bookmarked": false,43 "actions_summary": [],44 "moderator": false,45 "admin": false,46 "staff": false,47 "user_id": 29309,48 "hidden": false,49 "trust_level": 1,50 "deleted_at": null,51 "user_deleted": false,52 "edit_reason": null,53 "can_view_edit_history": true,54 "wiki": false,55 "post_url": "/t/pickup-a-loss-function-for-maximizing-pos-x-while-minimizing-neg-x/93684/1",56 "can_accept_answer": false,57 "can_unaccept_answer": false,58 "accepted_answer": false,59 "topic_accepted_answer": null,60 "can_vote": false61 }62 ],63 "stream": [64 22303965 ]66 },67 "timeline_lookup": [68 [69 1,70 189171 ]72 ],73 "suggested_topics": [74 {75 "fancy_title": "torch.OutOfMemoryError: CUDA out of memory",76 "id": 217669,77 "title": "torch.OutOfMemoryError: CUDA out of memory",78 "slug": "torch-outofmemoryerror-cuda-out-of-memory",79 "posts_count": 4,80 "reply_count": 2,81 "highest_post_number": 4,82 "image_url": null,83 "created_at": "2025-03-10T14:46:11.148Z",84 "last_posted_at": "2025-03-12T12:34:35.945Z",85 "bumped": true,86 "bumped_at": "2025-03-12T12:34:35.945Z",87 "archetype": "regular",88 "unseen": false,89 "pinned": false,90 "unpinned": null,91 "visible": true,92 "closed": false,93 "archived": false,94 "bookmarked": null,95 "liked": null,96 "tags_descriptions": {},97 "like_count": 0,98 "views": 580,99 "category_id": 1,100 "featured_link": null,101 "has_accepted_answer": false,102 "posters": [103 {104 "extras": "latest",105 "description": "Original Poster, Most Recent Poster",106 "user": {107 "id": 82794,108 "username": "Mohan_Krishna",109 "name": "Mohan Krishna",110 "avatar_template": "/user_avatar/discuss.pytorch.org/mohan_krishna/{size}/75758_2.png",111 "trust_level": 1112 }113 },114 {115 "extras": null,116 "description": "Frequent Poster",117 "user": {118 "id": 3534,119 "username": "ptrblck",120 "name": "",121 "avatar_template": "/user_avatar/discuss.pytorch.org/ptrblck/{size}/1823_2.png",122 "admin": true,123 "moderator": true,124 "trust_level": 2125 }126 }127 ]128 },129 {130 "fancy_title": "This is about optimizing cuDNN to avoid rebuilding the graph when batch, input dimensions change.",131 "id": 217915,132 "title": "This is about optimizing cuDNN to avoid rebuilding the graph when batch, input dimensions change.",133 "slug": "this-is-about-optimizing-cudnn-to-avoid-rebuilding-the-graph-when-batch-input-dimensions-change",134 "posts_count": 1,135 "reply_count": 0,136 "highest_post_number": 1,137 "image_url": null,138 "created_at": "2025-03-16T13:59:44.870Z",139 "last_posted_at": "2025-03-16T13:59:44.902Z",140 "bumped": true,141 "bumped_at": "2025-03-16T13:59:44.902Z",142 "archetype": "regular",143 "unseen": false,144 "pinned": false,145 "unpinned": null,146 "visible": true,147 "closed": false,148 "archived": false,149 "bookmarked": null,150 "liked": null,151 "tags_descriptions": {},152 "like_count": 0,153 "views": 36,154 "category_id": 1,155 "featured_link": null,156 "has_accepted_answer": false,157 "posters": [158 {159 "extras": "latest single",160 "description": "Original Poster, Most Recent Poster",161 "user": {162 "id": 83303,163 "username": "yhyang201",164 "name": "",165 "avatar_template": "/user_avatar/discuss.pytorch.org/yhyang201/{size}/76189_2.png",166 "trust_level": 0167 }168 }169 ]170 },171 {172 "fancy_title": "Why does not my code cause the error ‘element 0 of tensors does not require grad and does not have a grad_fn’",173 "id": 218268,174 "title": "Why does not my code cause the error 'element 0 of tensors does not require grad and does not have a grad_fn'",175 "slug": "why-does-not-my-code-cause-the-error-element-0-of-tensors-does-not-require-grad-and-does-not-have-a-grad-fn",176 "posts_count": 2,177 "reply_count": 0,178 "highest_post_number": 2,179 "image_url": null,180 "created_at": "2025-03-26T02:15:00.792Z",181 "last_posted_at": "2025-03-26T02:29:45.907Z",182 "bumped": true,183 "bumped_at": "2025-03-26T02:29:45.907Z",184 "archetype": "regular",185 "unseen": false,186 "pinned": false,187 "unpinned": null,188 "visible": true,189 "closed": false,190 "archived": false,191 "bookmarked": null,192 "liked": null,193 "tags_descriptions": {},194 "like_count": 0,195 "views": 45,196 "category_id": 1,197 "featured_link": null,198 "has_accepted_answer": true,199 "posters": [200 {201 "extras": null,202 "description": "Original Poster",203 "user": {204 "id": 82407,205 "username": "liao_zhang",206 "name": "liao zhang",207 "avatar_template": "/user_avatar/discuss.pytorch.org/liao_zhang/{size}/75390_2.png",208 "trust_level": 1209 }210 },211 {212 "extras": "latest",213 "description": "Most Recent Poster, Accepted Answer",214 "user": {215 "id": 3534,216 "username": "ptrblck",217 "name": "",218 "avatar_template": "/user_avatar/discuss.pytorch.org/ptrblck/{size}/1823_2.png",219 "admin": true,220 "moderator": true,221 "trust_level": 2222 }223 }224 ]225 },226 {227 "fancy_title": "Loaded checkpoint giving constant output during validation",228 "id": 218378,229 "title": "Loaded checkpoint giving constant output during validation",230 "slug": "loaded-checkpoint-giving-constant-output-during-validation",231 "posts_count": 2,232 "reply_count": 0,233 "highest_post_number": 2,234 "image_url": null,235 "created_at": "2025-03-28T20:32:18.592Z",236 "last_posted_at": "2025-03-28T23:29:50.092Z",237 "bumped": true,238 "bumped_at": "2025-03-28T23:29:50.092Z",239 "archetype": "regular",240 "unseen": false,241 "pinned": false,242 "unpinned": null,243 "visible": true,244 "closed": false,245 "archived": false,246 "bookmarked": null,247 "liked": null,248 "tags_descriptions": {},249 "like_count": 1,250 "views": 41,251 "category_id": 1,252 "featured_link": null,253 "has_accepted_answer": false,254 "posters": [255 {256 "extras": null,257 "description": "Original Poster",258 "user": {259 "id": 82259,260 "username": "anirudh_puligandla",261 "name": "anirudh puligandla",262 "avatar_template": "/user_avatar/discuss.pytorch.org/anirudh_puligandla/{size}/75256_2.png",263 "trust_level": 1264 }265 },266 {267 "extras": "latest",268 "description": "Most Recent Poster",269 "user": {270 "id": 83519,271 "username": "Alex_17127",272 "name": "Alex_17127",273 "avatar_template": "/user_avatar/discuss.pytorch.org/alex_17127/{size}/76386_2.png",274 "trust_level": 1275 }276 }277 ]278 },279 {280 "fancy_title": "Using Tensor subclasses for chunked loss and backprop?",281 "id": 212525,282 "title": "Using Tensor subclasses for chunked loss and backprop?",283 "slug": "using-tensor-subclasses-for-chunked-loss-and-backprop",284 "posts_count": 2,285 "reply_count": 0,286 "highest_post_number": 2,287 "image_url": null,288 "created_at": "2024-11-04T22:53:06.246Z",289 "last_posted_at": "2024-12-06T15:41:33.679Z",290 "bumped": true,291 "bumped_at": "2024-12-06T15:41:33.679Z",292 "archetype": "regular",293 "unseen": false,294 "pinned": false,295 "unpinned": null,296 "visible": true,297 "closed": false,298 "archived": false,299 "bookmarked": null,300 "liked": null,301 "tags_descriptions": {},302 "like_count": 0,303 "views": 198,304 "category_id": 1,305 "featured_link": null,306 "has_accepted_answer": false,307 "posters": [308 {309 "extras": null,310 "description": "Original Poster",311 "user": {312 "id": 8274,313 "username": "cbcase",314 "name": "Carl C",315 "avatar_template": "/user_avatar/discuss.pytorch.org/cbcase/{size}/61078_2.png",316 "trust_level": 1317 }318 },319 {320 "extras": "latest",321 "description": "Most Recent Poster",322 "user": {323 "id": 211,324 "username": "albanD",325 "name": "Alban D",326 "avatar_template": "/user_avatar/discuss.pytorch.org/alband/{size}/215_2.png",327 "admin": true,328 "moderator": true,329 "trust_level": 4330 }331 }332 ]333 }334 ],335 "tags_descriptions": {},336 "fancy_title": "Pickup a loss function for maximizing pos_x while minimizing neg_x",337 "id": 93684,338 "title": "Pickup a loss function for maximizing pos_x while minimizing neg_x",339 "posts_count": 1,340 "created_at": "2020-08-22T00:19:45.867Z",341 "views": 259,342 "reply_count": 0,343 "like_count": 0,344 "last_posted_at": "2020-08-22T00:19:45.919Z",345 "visible": true,346 "closed": false,347 "archived": false,348 "has_summary": false,349 "archetype": "regular",350 "slug": "pickup-a-loss-function-for-maximizing-pos-x-while-minimizing-neg-x",351 "category_id": 1,352 "word_count": 165,353 "deleted_at": null,354 "user_id": 29309,355 "featured_link": null,356 "pinned_globally": false,357 "pinned_at": null,358 "pinned_until": null,359 "image_url": null,360 "slow_mode_seconds": 0,361 "draft": null,362 "draft_key": "topic_93684",363 "draft_sequence": null,364 "unpinned": null,365 "pinned": false,366 "current_post_number": 1,367 "highest_post_number": 1,368 "deleted_by": null,369 "actions_summary": [370 {371 "id": 4,372 "count": 0,373 "hidden": false,374 "can_act": false375 },376 {377 "id": 8,378 "count": 0,379 "hidden": false,380 "can_act": false381 },382 {383 "id": 10,384 "count": 0,385 "hidden": false,386 "can_act": false387 },388 {389 "id": 7,390 "count": 0,391 "hidden": false,392 "can_act": false393 }394 ],395 "chunk_size": 20,396 "bookmarked": false,397 "topic_timer": null,398 "message_bus_last_id": 0,399 "participant_count": 1,400 "show_read_indicator": false,401 "thumbnails": null,402 "slow_mode_enabled_until": null,403 "can_vote": false,404 "vote_count": 0,405 "user_voted": false,406 "discourse_zendesk_plugin_zendesk_id": null,407 "discourse_zendesk_plugin_zendesk_url": "https://your-url.zendesk.com/agent/tickets/",408 "details": {409 "can_edit": false,410 "notification_level": 1,411 "participants": [412 {413 "id": 29309,414 "username": "themoonboy",415 "name": "",416 "avatar_template": "/user_avatar/discuss.pytorch.org/themoonboy/{size}/13133_2.png",417 "post_count": 1,418 "primary_group_name": null,419 "flair_name": null,420 "flair_url": null,421 "flair_color": null,422 "flair_bg_color": null,423 "flair_group_id": null,424 "trust_level": 1425 }426 ],427 "created_by": {428 "id": 29309,429 "username": "themoonboy",430 "name": "",431 "avatar_template": "/user_avatar/discuss.pytorch.org/themoonboy/{size}/13133_2.png"432 },433 "last_poster": {434 "id": 29309,435 "username": "themoonboy",436 "name": "",437 "avatar_template": "/user_avatar/discuss.pytorch.org/themoonboy/{size}/13133_2.png"438 }439 },440 "bookmarks": []441 },442 {443 "post_stream": {444 "posts": [445 {446 "id": 223025,447 "name": "Em Ros",448 "username": "Em_Ros",449 "avatar_template": "/user_avatar/discuss.pytorch.org/em_ros/{size}/28032_2.png",450 "created_at": "2020-08-21T22:55:30.779Z",451 "cooked": "<p>Hello!</p>\n<p>I want to implement a custom loss function that is a sum of log linear loss (for binary classification).</p>\n<p>I tried digging in the forum and in general for a solution but I can’t seem to find how to do it. Could you maybe point me towards a source for this?</p>\n<p>Thanks!</p>",452 "post_number": 1,453 "post_type": 1,454 "posts_count": 1,455 "updated_at": "2020-08-21T22:55:30.779Z",456 "reply_count": 0,457 "reply_to_post_number": null,458 "quote_count": 0,459 "incoming_link_count": 11,460 "reads": 6,461 "readers_count": 5,462 "score": 56.2,463 "yours": false,464 "topic_id": 93678,465 "topic_slug": "joint-learning-log-linear-models",466 "display_username": "Em Ros",467 "primary_group_name": null,468 "flair_name": null,469 "flair_url": null,470 "flair_bg_color": null,471 "flair_color": null,472 "flair_group_id": null,473 "badges_granted": [],474 "version": 1,475 "can_edit": false,476 "can_delete": false,477 "can_recover": false,478 "can_see_hidden_post": false,479 "can_wiki": false,480 "read": true,481 "user_title": null,482 "bookmarked": false,483 "actions_summary": [],484 "moderator": false,485 "admin": false,486 "staff": false,487 "user_id": 35809,488 "hidden": false,489 "trust_level": 1,490 "deleted_at": null,491 "user_deleted": false,492 "edit_reason": null,493 "can_view_edit_history": true,494 "wiki": false,495 "post_url": "/t/joint-learning-log-linear-models/93678/1",496 "can_accept_answer": false,497 "can_unaccept_answer": false,498 "accepted_answer": false,499 "topic_accepted_answer": null,500 "can_vote": false501 }502 ],503 "stream": [504 223025505 ]506 },507 "timeline_lookup": [508 [509 1,510 1891511 ]512 ],513 "suggested_topics": [514 {515 "fancy_title": "Pytorch import leads to libcudnn error",516 "id": 216791,517 "title": "Pytorch import leads to libcudnn error",518 "slug": "pytorch-import-leads-to-libcudnn-error",519 "posts_count": 8,520 "reply_count": 5,521 "highest_post_number": 8,522 "image_url": "https://discuss.pytorch.org/uploads/default/optimized/3X/f/0/f07cd098193678b9b0ce96eecebafa317b8c68d7_2_1024x174.png",523 "created_at": "2025-02-17T22:08:55.963Z",524 "last_posted_at": "2025-02-18T18:23:36.720Z",525 "bumped": true,526 "bumped_at": "2025-02-18T18:23:36.720Z",527 "archetype": "regular",528 "unseen": false,529 "pinned": false,530 "unpinned": null,531 "visible": true,532 "closed": false,533 "archived": false,534 "bookmarked": null,535 "liked": null,536 "tags_descriptions": {},537 "like_count": 0,538 "views": 1846,539 "category_id": 1,540 "featured_link": null,541 "has_accepted_answer": false,542 "posters": [543 {544 "extras": null,545 "description": "Original Poster",546 "user": {547 "id": 82746,548 "username": "SimSim3097",549 "name": "",550 "avatar_template": "/user_avatar/discuss.pytorch.org/simsim3097/{size}/72488_2.png",551 "trust_level": 1552 }553 },554 {555 "extras": "latest",556 "description": "Most Recent Poster",557 "user": {558 "id": 3534,559 "username": "ptrblck",560 "name": "",561 "avatar_template": "/user_avatar/discuss.pytorch.org/ptrblck/{size}/1823_2.png",562 "admin": true,563 "moderator": true,564 "trust_level": 2565 }566 }567 ]568 },569 {570 "fancy_title": "How to fix profiler “assert not duplicate” error?",571 "id": 214829,572 "title": "How to fix profiler \"assert not duplicate\" error?",573 "slug": "how-to-fix-profiler-assert-not-duplicate-error",574 "posts_count": 4,575 "reply_count": 2,576 "highest_post_number": 4,577 "image_url": "https://discuss.pytorch.org/uploads/default/optimized/3X/3/f/3fc597ec1e5259a7157122d19ef9c1ca3e7e46c5_2_1024x288.png",578 "created_at": "2024-12-31T15:34:27.064Z",579 "last_posted_at": "2025-01-05T20:45:11.034Z",580 "bumped": true,581 "bumped_at": "2025-01-05T20:45:11.034Z",582 "archetype": "regular",583 "unseen": false,584 "pinned": false,585 "unpinned": null,586 "visible": true,587 "closed": false,588 "archived": false,589 "bookmarked": null,590 "liked": null,591 "tags_descriptions": {},592 "like_count": 0,593 "views": 226,594 "category_id": 1,595 "featured_link": null,596 "has_accepted_answer": false,597 "posters": [598 {599 "extras": null,600 "description": "Original Poster",601 "user": {602 "id": 81811,603 "username": "Anh_Pham",604 "name": "Anh Pham",605 "avatar_template": "/user_avatar/discuss.pytorch.org/anh_pham/{size}/74836_2.png",606 "trust_level": 0607 }608 },609 {610 "extras": "latest",611 "description": "Most Recent Poster",612 "user": {613 "id": 41396,614 "username": "soulitzer",615 "name": "",616 "avatar_template": "/letter_avatar_proxy/v4/letter/s/839c29/{size}.png",617 "trust_level": 2618 }619 }620 ]621 },622 {623 "fancy_title": "Reduce memory usage during iterative algorithm",624 "id": 215646,625 "title": "Reduce memory usage during iterative algorithm",626 "slug": "reduce-memory-usage-during-iterative-algorithm",627 "posts_count": 2,628 "reply_count": 0,629 "highest_post_number": 3,630 "image_url": null,631 "created_at": "2025-01-21T05:40:14.862Z",632 "last_posted_at": "2025-01-22T08:25:39.660Z",633 "bumped": true,634 "bumped_at": "2025-01-22T08:25:39.660Z",635 "archetype": "regular",636 "unseen": false,637 "pinned": false,638 "unpinned": null,639 "visible": true,640 "closed": false,641 "archived": false,642 "bookmarked": null,643 "liked": null,644 "tags_descriptions": {},645 "like_count": 0,646 "views": 149,647 "category_id": 1,648 "featured_link": null,649 "has_accepted_answer": false,650 "posters": [651 {652 "extras": null,653 "description": "Original Poster",654 "user": {655 "id": 72757,656 "username": "sherlock.h",657 "name": "",658 "avatar_template": "/user_avatar/discuss.pytorch.org/sherlock.h/{size}/62377_2.png",659 "trust_level": 1660 }661 },662 {663 "extras": "latest",664 "description": "Most Recent Poster",665 "user": {666 "id": 68149,667 "username": "Soumya_Kundu",668 "name": "Soumya Snigdha Kundu",669 "avatar_template": "/user_avatar/discuss.pytorch.org/soumya_kundu/{size}/71716_2.png",670 "trust_level": 2671 }672 }673 ]674 },675 {676 "fancy_title": "Conda Pytorch keeps downgrading itself from cuda -> cpu",677 "id": 216525,678 "title": "Conda Pytorch keeps downgrading itself from cuda -> cpu",679 "slug": "conda-pytorch-keeps-downgrading-itself-from-cuda-cpu",680 "posts_count": 4,681 "reply_count": 1,682 "highest_post_number": 4,683 "image_url": null,684 "created_at": "2025-02-11T10:46:19.656Z",685 "last_posted_at": "2025-02-11T20:21:44.809Z",686 "bumped": true,687 "bumped_at": "2025-02-11T20:21:44.809Z",688 "archetype": "regular",689 "unseen": false,690 "pinned": false,691 "unpinned": null,692 "visible": true,693 "closed": false,694 "archived": false,695 "bookmarked": null,696 "liked": null,697 "tags_descriptions": {},698 "like_count": 0,699 "views": 251,700 "category_id": 1,701 "featured_link": null,702 "has_accepted_answer": false,703 "posters": [704 {705 "extras": null,706 "description": "Original Poster",707 "user": {708 "id": 71072,709 "username": "wilhelm",710 "name": "Dave",711 "avatar_template": "/user_avatar/discuss.pytorch.org/wilhelm/{size}/70280_2.png",712 "trust_level": 2713 }714 },715 {716 "extras": "latest",717 "description": "Most Recent Poster",718 "user": {719 "id": 3534,720 "username": "ptrblck",721 "name": "",722 "avatar_template": "/user_avatar/discuss.pytorch.org/ptrblck/{size}/1823_2.png",723 "admin": true,724 "moderator": true,725 "trust_level": 2726 }727 }728 ]729 },730 {731 "fancy_title": "Pruning by manipulating weight_mask",732 "id": 216953,733 "title": "Pruning by manipulating weight_mask",734 "slug": "pruning-by-manipulating-weight-mask",735 "posts_count": 2,736 "reply_count": 0,737 "highest_post_number": 2,738 "image_url": null,739 "created_at": "2025-02-20T14:21:28.443Z",740 "last_posted_at": "2025-03-15T22:52:11.716Z",741 "bumped": true,742 "bumped_at": "2025-03-15T22:52:11.716Z",743 "archetype": "regular",744 "unseen": false,745 "pinned": false,746 "unpinned": null,747 "visible": true,748 "closed": false,749 "archived": false,750 "bookmarked": null,751 "liked": null,752 "tags_descriptions": {},753 "like_count": 0,754 "views": 75,755 "category_id": 1,756 "featured_link": null,757 "has_accepted_answer": false,758 "posters": [759 {760 "extras": "latest single",761 "description": "Original Poster, Most Recent Poster",762 "user": {763 "id": 520,764 "username": "ndronen",765 "name": "Nicholas Dronen",766 "avatar_template": "/user_avatar/discuss.pytorch.org/ndronen/{size}/1575_2.png",767 "trust_level": 2768 }769 }770 ]771 }772 ],773 "tags_descriptions": {},774 "fancy_title": "Joint learning log linear models",775 "id": 93678,776 "title": "Joint learning log linear models",777 "posts_count": 1,778 "created_at": "2020-08-21T22:55:30.720Z",779 "views": 310,780 "reply_count": 0,781 "like_count": 0,782 "last_posted_at": "2020-08-21T22:55:30.779Z",783 "visible": true,784 "closed": false,785 "archived": false,786 "has_summary": false,787 "archetype": "regular",788 "slug": "joint-learning-log-linear-models",789 "category_id": 1,790 "word_count": 54,791 "deleted_at": null,792 "user_id": 35809,793 "featured_link": null,794 "pinned_globally": false,795 "pinned_at": null,796 "pinned_until": null,797 "image_url": null,798 "slow_mode_seconds": 0,799 "draft": null,800 "draft_key": "topic_93678",801 "draft_sequence": null,802 "unpinned": null,803 "pinned": false,804 "current_post_number": 1,805 "highest_post_number": 1,806 "deleted_by": null,807 "actions_summary": [808 {809 "id": 4,810 "count": 0,811 "hidden": false,812 "can_act": false813 },814 {815 "id": 8,816 "count": 0,817 "hidden": false,818 "can_act": false819 },820 {821 "id": 10,822 "count": 0,823 "hidden": false,824 "can_act": false825 },826 {827 "id": 7,828 "count": 0,829 "hidden": false,830 "can_act": false831 }832 ],833 "chunk_size": 20,834 "bookmarked": false,835 "topic_timer": null,836 "message_bus_last_id": 0,837 "participant_count": 1,838 "show_read_indicator": false,839 "thumbnails": null,840 "slow_mode_enabled_until": null,841 "can_vote": false,842 "vote_count": 0,843 "user_voted": false,844 "discourse_zendesk_plugin_zendesk_id": null,845 "discourse_zendesk_plugin_zendesk_url": "https://your-url.zendesk.com/agent/tickets/",846 "details": {847 "can_edit": false,848 "notification_level": 1,849 "participants": [850 {851 "id": 35809,852 "username": "Em_Ros",853 "name": "Em Ros",854 "avatar_template": "/user_avatar/discuss.pytorch.org/em_ros/{size}/28032_2.png",855 "post_count": 1,856 "primary_group_name": null,857 "flair_name": null,858 "flair_url": null,859 "flair_color": null,860 "flair_bg_color": null,861 "flair_group_id": null,862 "trust_level": 1863 }864 ],865 "created_by": {866 "id": 35809,867 "username": "Em_Ros",868 "name": "Em Ros",869 "avatar_template": "/user_avatar/discuss.pytorch.org/em_ros/{size}/28032_2.png"870 },871 "last_poster": {872 "id": 35809,873 "username": "Em_Ros",874 "name": "Em Ros",875 "avatar_template": "/user_avatar/discuss.pytorch.org/em_ros/{size}/28032_2.png"876 }877 },878 "bookmarks": []879 },880 {881 "post_stream": {882 "posts": [883 {884 "id": 218287,885 "name": "",886 "username": "pkadambi",887 "avatar_template": "/letter_avatar_proxy/v4/letter/p/eb9ed0/{size}.png",888 "created_at": "2020-08-05T07:14:01.072Z",889 "cooked": "<p>I’m trying to quantize BERT to 4 bits or mixed precision, and I don’t see available methods to to quantization aware training on BERT for any precision other than torch.uint8. This is given in the dynamic quantization tutorial.<br>\nI want to use both post training quantization and dynamic quantization for lower than 8 bits.</p>\n<p>Will I have to rewrite the modeling_bert.py (transformers/modeling_bert.py) layers with fake quantization added? How can lower than 8bit precision and mixed precision be implemented on BERT?</p>",890 "post_number": 1,891 "post_type": 1,892 "posts_count": 5,893 "updated_at": "2020-08-05T08:08:07.927Z",894 "reply_count": 0,895 "reply_to_post_number": null,896 "quote_count": 0,897 "incoming_link_count": 401,898 "reads": 36,899 "readers_count": 35,900 "score": 2012.2,901 "yours": false,902 "topic_id": 91686,903 "topic_slug": "quantizing-transformer-architecture-below-8-bit-post-training-quantization",904 "display_username": "",905 "primary_group_name": null,906 "flair_name": null,907 "flair_url": null,908 "flair_bg_color": null,909 "flair_color": null,910 "flair_group_id": null,911 "badges_granted": [],912 "version": 2,913 "can_edit": false,914 "can_delete": false,915 "can_recover": false,916 "can_see_hidden_post": false,917 "can_wiki": false,918 "read": true,919 "user_title": null,920 "bookmarked": false,921 "actions_summary": [],922 "moderator": false,923 "admin": false,924 "staff": false,925 "user_id": 11592,926 "hidden": false,927 "trust_level": 1,928 "deleted_at": null,929 "user_deleted": false,930 "edit_reason": null,931 "can_view_edit_history": true,932 "wiki": false,933 "post_url": "/t/quantizing-transformer-architecture-below-8-bit-post-training-quantization/91686/1",934 "can_accept_answer": false,935 "can_unaccept_answer": false,936 "accepted_answer": false,937 "topic_accepted_answer": null,938 "can_vote": false939 },940 {941 "id": 218373,942 "name": "Thomas V",943 "username": "tom",944 "avatar_template": "/user_avatar/discuss.pytorch.org/tom/{size}/3162_2.png",945 "created_at": "2020-08-05T10:17:06.407Z",946 "cooked": "<p>The difficulty there is PyTorch inherently assumes that things are at least 1 byte when doing things with memory.<br>\nI’d probably convert to <a href=\"https://tvm.ai/\" rel=\"nofollow noopener\">TVM</a> and see what can be done there.<br>\n(QAT with fake quantization probably could work for 4 bits, too.)</p>",947 "post_number": 2,948 "post_type": 1,949 "posts_count": 5,950 "updated_at": "2020-08-05T12:22:04.553Z",951 "reply_count": 1,952 "reply_to_post_number": null,953 "quote_count": 0,954 "incoming_link_count": 4,955 "reads": 34,956 "readers_count": 33,957 "score": 31.8,958 "yours": false,959 "topic_id": 91686,960 "topic_slug": "quantizing-transformer-architecture-below-8-bit-post-training-quantization",961 "display_username": "Thomas V",962 "primary_group_name": null,963 "flair_name": null,964 "flair_url": null,965 "flair_bg_color": null,966 "flair_color": null,967 "flair_group_id": null,968 "badges_granted": [],969 "version": 2,970 "can_edit": false,971 "can_delete": false,972 "can_recover": false,973 "can_see_hidden_post": false,974 "can_wiki": false,975 "link_counts": [976 {977 "url": "https://tvm.ai/",978 "internal": false,979 "reflection": false,980 "title": "Redirecting…",981 "clicks": 22982 }983 ],984 "read": true,985 "user_title": null,986 "bookmarked": false,987 "actions_summary": [],988 "moderator": false,989 "admin": false,990 "staff": false,991 "user_id": 616,992 "hidden": false,993 "trust_level": 2,994 "deleted_at": null,995 "user_deleted": false,996 "edit_reason": null,997 "can_view_edit_history": true,998 "wiki": false,999 "post_url": "/t/quantizing-transformer-architecture-below-8-bit-post-training-quantization/91686/2",1000 "can_accept_answer": false,1001 "can_unaccept_answer": false,1002 "accepted_answer": false,1003 "topic_accepted_answer": null1004 },1005 {1006 "id": 218571,1007 "name": "",1008 "username": "pkadambi",1009 "avatar_template": "/letter_avatar_proxy/v4/letter/p/eb9ed0/{size}.png",1010 "created_at": "2020-08-06T04:04:16.518Z",1011 "cooked": "<p>It’s not an issue even if the weights are stored as FP32 values in memory.<br>\nI’m trying to evaluate post training quantization or fine tune the model with quantization aware training, but do this all under under fake quantization to any bit width of my choosing.</p>",1012 "post_number": 3,1013 "post_type": 1,1014 "posts_count": 5,1015 "updated_at": "2020-08-06T04:04:16.518Z",1016 "reply_count": 2,1017 "reply_to_post_number": 2,1018 "quote_count": 0,1019 "incoming_link_count": 1,1020 "reads": 34,1021 "readers_count": 33,1022 "score": 21.8,1023 "yours": false,1024 "topic_id": 91686,1025 "topic_slug": "quantizing-transformer-architecture-below-8-bit-post-training-quantization",1026 "display_username": "",1027 "primary_group_name": null,1028 "flair_name": null,1029 "flair_url": null,1030 "flair_bg_color": null,1031 "flair_color": null,1032 "flair_group_id": null,1033 "badges_granted": [],1034 "version": 1,1035 "can_edit": false,1036 "can_delete": false,1037 "can_recover": false,1038 "can_see_hidden_post": false,1039 "can_wiki": false,1040 "read": true,1041 "user_title": null,1042 "reply_to_user": {1043 "id": 616,1044 "username": "tom",1045 "name": "Thomas V",1046 "avatar_template": "/user_avatar/discuss.pytorch.org/tom/{size}/3162_2.png"1047 },1048 "bookmarked": false,1049 "actions_summary": [],1050 "moderator": false,1051 "admin": false,1052 "staff": false,1053 "user_id": 11592,1054 "hidden": false,1055 "trust_level": 1,1056 "deleted_at": null,1057 "user_deleted": false,1058 "edit_reason": null,1059 "can_view_edit_history": true,1060 "wiki": false,1061 "post_url": "/t/quantizing-transformer-architecture-below-8-bit-post-training-quantization/91686/3",1062 "can_accept_answer": false,1063 "can_unaccept_answer": false,1064 "accepted_answer": false,1065 "topic_accepted_answer": null1066 },1067 {1068 "id": 219855,1069 "name": "Thomas V",1070 "username": "tom",1071 "avatar_template": "/user_avatar/discuss.pytorch.org/tom/{size}/3162_2.png",1072 "created_at": "2020-08-11T02:23:30.311Z",1073 "cooked": "<p>While I don’t think it works out of the box, you could try to adapt the observers and fake quant layers to be more flexible. For example, there are some obvious 8 bit hard coded values here:<br>\n</p><aside class=\"onebox githubblob\">\n <header class=\"source\">\n <a href=\"https://github.com/pytorch/pytorch/blob/a414bd69de8d01af44751bfe327703ec997dafd9/torch/quantization/observer.py#L146\" target=\"_blank\" rel=\"nofollow noopener\">github.com</a>\n </header>\n <article class=\"onebox-body\">\n <h4><a href=\"https://github.com/pytorch/pytorch/blob/a414bd69de8d01af44751bfe327703ec997dafd9/torch/quantization/observer.py#L146\" target=\"_blank\" rel=\"nofollow noopener\">pytorch/pytorch/blob/a414bd69de8d01af44751bfe327703ec997dafd9/torch/quantization/observer.py#L146</a></h4>\n<pre class=\"onebox\"><code class=\"lang-py\"><ol class=\"start lines\" start=\"136\" style=\"counter-reset: li-counter 135 ;\">\n<li> Learned Step Size Quantization: https://openreview.net/pdf?id=rkgO66VKDS</li>\n<li> Trained Quantization Thresholds: https://arxiv.org/pdf/1903.08066.pdf</li>\n<li> \"\"\"</li>\n<li> # The variable names are prefixed with \"initial\" because their values (qmin and qmax) might be adjusted</li>\n<li> # based on whether quantization range is reduced and the datatype (signed/unsigned) used by the observer.</li>\n<li> initial_qmin, initial_qmax = initial_dynamic_qrange</li>\n<li> assert initial_qmin <= 0 <= initial_qmax, \"Dynamic quantization range must include 0.\"</li>\n<li> assert initial_qmin < initial_qmax, \"qmin must be strictly less than qmax for dynamic quantization range.\"</li>\n<li>\n</li><li>@torch.jit.export</li>\n<li class=\"selected\">def _calculate_qmin_qmax(self):</li>\n<li> # type: () -> Tuple[int, int]</li>\n<li> r\"\"\"Calculates actual qmin and qmax based on the quantization range,</li>\n<li> observer datatype and if range is reduced.</li>\n<li> \"\"\"</li>\n<li> if self.is_dynamic_qrange:</li>\n<li> # This initialization here is to be resolve TorchScript compilation issues and allow</li>\n<li> # using of refinement to decouple initial_qmin and initial_qmax from quantization range.</li>\n<li> # The actual values of initial_qmin and initial_qmax will be reset below.</li>\n<li> initial_qmin, initial_qmax = 0, 255</li>\n<li> # The following assignment of initial_qrange to a local variable and the if check refine the</li>\n</ol></code></pre>\n\n\n </article>\n <div class=\"onebox-metadata\">\n \n \n </div>\n <div style=\"clear: both\"></div>\n</aside>\n",1074 "post_number": 4,1075 "post_type": 1,1076 "posts_count": 5,1077 "updated_at": "2020-08-11T02:23:30.311Z",1078 "reply_count": 1,1079 "reply_to_post_number": 3,1080 "quote_count": 0,1081 "incoming_link_count": 20,1082 "reads": 28,1083 "readers_count": 27,1084 "score": 110.6,1085 "yours": false,1086 "topic_id": 91686,1087 "topic_slug": "quantizing-transformer-architecture-below-8-bit-post-training-quantization",1088 "display_username": "Thomas V",1089 "primary_group_name": null,1090 "flair_name": null,1091 "flair_url": null,1092 "flair_bg_color": null,1093 "flair_color": null,1094 "flair_group_id": null,1095 "badges_granted": [],1096 "version": 1,1097 "can_edit": false,1098 "can_delete": false,1099 "can_recover": false,1100 "can_see_hidden_post": false,1101 "can_wiki": false,1102 "link_counts": [1103 {1104 "url": "https://github.com/pytorch/pytorch/blob/a414bd69de8d01af44751bfe327703ec997dafd9/torch/quantization/observer.py#L146",1105 "internal": false,1106 "reflection": false,1107 "title": "pytorch/observer.py at a414bd69de8d01af44751bfe327703ec997dafd9 · pytorch/pytorch · GitHub",1108 "clicks": 51109 }1110 ],1111 "read": true,1112 "user_title": null,1113 "reply_to_user": {1114 "id": 11592,1115 "username": "pkadambi",1116 "name": "",1117 "avatar_template": "/letter_avatar_proxy/v4/letter/p/eb9ed0/{size}.png"1118 },1119 "bookmarked": false,1120 "actions_summary": [],1121 "moderator": false,1122 "admin": false,1123 "staff": false,1124 "user_id": 616,1125 "hidden": false,1126 "trust_level": 2,1127 "deleted_at": null,1128 "user_deleted": false,1129 "edit_reason": null,1130 "can_view_edit_history": true,1131 "wiki": false,1132 "post_url": "/t/quantizing-transformer-architecture-below-8-bit-post-training-quantization/91686/4",1133 "can_accept_answer": false,1134 "can_unaccept_answer": false,1135 "accepted_answer": false,1136 "topic_accepted_answer": null1137 },1138 {1139 "id": 223024,1140 "name": "Jerry Zhang",1141 "username": "jerryzh168",1142 "avatar_template": "/user_avatar/discuss.pytorch.org/jerryzh168/{size}/15217_2.png",1143 "created_at": "2020-08-21T22:54:43.028Z",1144 "cooked": "<aside class=\"quote no-group\" data-username=\"pkadambi\" data-post=\"3\" data-topic=\"91686\">\n<div class=\"title\">\n<div class=\"quote-controls\"></div>\n<img loading=\"lazy\" alt=\"\" width=\"24\" height=\"24\" src=\"https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/p/eb9ed0/48.png\" class=\"avatar\"> pkadambi:</div>\n<blockquote>\n<p>I’m trying to evaluate post training quantization or fine tune the model with quantization aware training, but do this all under under fake quantization to any bit width of my choosing.</p>\n</blockquote>\n</aside>\n<p>we do have the support for lower bits in <a href=\"https://github.com/pytorch/pytorch/blob/master/torch/quantization/observer.py#L185\" rel=\"noopener nofollow ugc\">https://github.com/pytorch/pytorch/blob/master/torch/quantization/observer.py#L185</a> now, one of our interns just added this recently.</p>",1145 "post_number": 5,1146 "post_type": 1,1147 "posts_count": 5,1148 "updated_at": "2020-08-21T22:54:43.028Z",1149 "reply_count": 0,1150 "reply_to_post_number": 4,1151 "quote_count": 1,1152 "incoming_link_count": 10,1153 "reads": 23,1154 "readers_count": 22,1155 "score": 69.6,1156 "yours": false,1157 "topic_id": 91686,1158 "topic_slug": "quantizing-transformer-architecture-below-8-bit-post-training-quantization",1159 "display_username": "Jerry Zhang",1160 "primary_group_name": null,1161 "flair_name": null,1162 "flair_url": null,1163 "flair_bg_color": null,1164 "flair_color": null,1165 "flair_group_id": null,1166 "badges_granted": [],1167 "version": 1,1168 "can_edit": false,1169 "can_delete": false,1170 "can_recover": false,1171 "can_see_hidden_post": false,1172 "can_wiki": false,1173 "link_counts": [1174 {1175 "url": "https://github.com/pytorch/pytorch/blob/master/torch/quantization/observer.py#L185",1176 "internal": false,1177 "reflection": false,1178 "title": "pytorch/observer.py at master · pytorch/pytorch · GitHub",1179 "clicks": 451180 }1181 ],1182 "read": true,1183 "user_title": null,1184 "reply_to_user": {1185 "id": 616,1186 "username": "tom",1187 "name": "Thomas V",1188 "avatar_template": "/user_avatar/discuss.pytorch.org/tom/{size}/3162_2.png"1189 },1190 "bookmarked": false,1191 "actions_summary": [1192 {1193 "id": 2,1194 "count": 11195 }1196 ],1197 "moderator": false,1198 "admin": false,1199 "staff": false,1200 "user_id": 21770,