Anurag1734/cuda-error-resolution-analysis
07
1[2 {3 "post_stream": {4 "posts": [5 {6 "id": 428831,7 "name": "Martin Nguyen",8 "username": "Martin_Nguyen",9 "avatar_template": "/user_avatar/discuss.pytorch.org/martin_nguyen/{size}/61717_2.png",10 "created_at": "2024-01-02T01:30:10.940Z",11 "cooked": "<p>Greetings! I am using Bayesian Neural Network and pytorch randomizing/sampling keeps yielding nan in some epoch. Does anyone know how to fix this? I have tried:</p>\n<p>Trial 1: <code>torch.empty(120).uniform_(-.01, 0.01)</code><br>\nTrial 2: <code>torch.randn(120).uniform_(-.01, 0.01)</code><br>\nTrial 3: <code>torch.distributions.Uniform(-.01, 0.01).sample((1, 120))[0]</code><br>\nTrial 4: <code>torch.torch.distributions.Uniform(-.01, 0.01).sample((1, 120))[0]</code></p>\n<p>I also even tried like this for all the above trials <code>torch.nan_to_num(torch.torch.distributions.Uniform(-.01, 0.01).sample((1, out_features))[0], nan=torch.tensor(0.0037))</code>, but this is not gonna work in the training loop. In particular, it only took into account the part <code>torch.torch.distributions.Uniform(-.01, 0.01).sample((1, out_features))[0]</code> and still yielded <code>nan</code>.</p>\n<p>The results I got with nan might not be reproducible with the examples of trials I gave here. Please let me know if you need to know deeper to reproduce the results with nan.</p>\n<p>This is the result in some epoch I tried with <code>torch.torch.distributions.Uniform(-.01, 0.01).sample((1, 120))[0]</code> and yielded <code>nan</code></p>\n<pre><code class=\"lang-auto\">tensor([-1.4659, 0.3137, 0.1822, 0.1657, 0.2099, -0.1929, 0.2386, 0.4739,\n 0.2335, 0.2515, 0.2949, 0.2874, 0.1460, -2.9367, 0.6829, 0.2574,\n 0.4183, 0.3023, 0.4242, -1.0063, 0.3041, nan, 0.2051, 0.1333,\n 0.3006, 0.2397, 0.2347, 0.1516, 0.3593, 0.2641, 0.1562, -0.5621,\n -0.2820, 0.3090, 0.1144, 0.5003, 0.3033, 0.1153, 0.3229, 0.4727,\n 0.0221, 0.2196, 0.3357, 0.1594, 0.3025, -0.3293, 0.0314, 0.1627,\n 0.0971, -0.7528, 0.3589, 0.2749, 0.3002, 0.1839, -0.5211, -0.5340,\n 0.2862, 0.3999, 0.1999, 0.0734, 0.3181, 0.1325, -0.6105, 0.2866,\n 0.2635, 0.2449, 0.1772, 0.2520, -4.4859, 0.4437, 0.2694, 0.1909,\n 0.4727, 0.2047, 0.3235, 0.4658, -1.2267, 0.1527, 0.0942, 0.5141,\n 0.6169, 0.2492, 0.0831, 0.0781, 0.0855, 0.1781, 0.1907, 0.5520,\n 0.3903, 0.1470, 0.3136, 0.3869, -7.8619, 0.1830, 0.1720, 0.3044,\n 0.1460, 0.3055, 0.1487, -0.0105, -2.5014, 0.0465, -0.2051, 0.2039,\n -4.9786, 0.3331, 0.3496, -0.0930, 0.2421, -3.2148, 0.2964, 0.2464,\n 0.3046, 0.2449, 0.2755, 0.3433, 0.3893, 0.3142, 0.0477, 0.1953])\n</code></pre>\n<p>The versions of packages relating to pytorch are here:</p>\n<p><img src=\"https://discuss.pytorch.org/uploads/default/original/3X/a/6/a679cdf2aa7ca749d997c509a9dc221568bfa1ad.png\" alt=\"Pytorch asking\" data-base62-sha1=\"nKIdWTkCj6o5ahSYpYsG3FgKGZf\" width=\"528\" height=\"99\"></p>\n<p>It has already taken me nearly a month to find solutions for this but still not successful! So I guess I should ask for help here. Please help me with this. Thank you very much.</p>",12 "post_number": 1,13 "post_type": 1,14 "posts_count": 5,15 "updated_at": "2024-01-02T01:43:42.307Z",16 "reply_count": 0,17 "reply_to_post_number": null,18 "quote_count": 0,19 "incoming_link_count": 30,20 "reads": 7,21 "readers_count": 6,22 "score": 151.4,23 "yours": false,24 "topic_id": 194678,25 "topic_slug": "how-to-fix-when-pytorch-randomizing-sampling-keeps-yielding-nan",26 "display_username": "Martin Nguyen",27 "primary_group_name": null,28 "flair_name": null,29 "flair_url": null,30 "flair_bg_color": null,31 "flair_color": null,32 "flair_group_id": null,33 "badges_granted": [],34 "version": 3,35 "can_edit": false,36 "can_delete": false,37 "can_recover": false,38 "can_see_hidden_post": false,39 "can_wiki": false,40 "read": true,41 "user_title": null,42 "bookmarked": false,43 "actions_summary": [],44 "moderator": false,45 "admin": false,46 "staff": false,47 "user_id": 72161,48 "hidden": false,49 "trust_level": 0,50 "deleted_at": null,51 "user_deleted": false,52 "edit_reason": null,53 "can_view_edit_history": true,54 "wiki": false,55 "post_url": "/t/how-to-fix-when-pytorch-randomizing-sampling-keeps-yielding-nan/194678/1",56 "can_accept_answer": false,57 "can_unaccept_answer": false,58 "accepted_answer": false,59 "topic_accepted_answer": null,60 "can_vote": false61 },62 {63 "id": 428973,64 "name": "Alban D",65 "username": "albanD",66 "avatar_template": "/user_avatar/discuss.pytorch.org/alband/{size}/215_2.png",67 "created_at": "2024-01-03T08:53:25.779Z",68 "cooked": "<p>Hi,</p>\n<p>A quick clarifying question: is it the sampling function itself that yields nan or your network or your loss function?<br>\nI would suggest to first figure out which op exactly is generating such nan by adding checks within the network.</p>",69 "post_number": 2,70 "post_type": 1,71 "posts_count": 5,72 "updated_at": "2024-01-03T08:53:25.779Z",73 "reply_count": 1,74 "reply_to_post_number": null,75 "quote_count": 0,76 "incoming_link_count": 0,77 "reads": 7,78 "readers_count": 6,79 "score": 6.4,80 "yours": false,81 "topic_id": 194678,82 "topic_slug": "how-to-fix-when-pytorch-randomizing-sampling-keeps-yielding-nan",83 "display_username": "Alban D",84 "primary_group_name": null,85 "flair_name": null,86 "flair_url": null,87 "flair_bg_color": null,88 "flair_color": null,89 "flair_group_id": null,90 "badges_granted": [],91 "version": 1,92 "can_edit": false,93 "can_delete": false,94 "can_recover": false,95 "can_see_hidden_post": false,96 "can_wiki": false,97 "read": true,98 "user_title": "",99 "bookmarked": false,100 "actions_summary": [],101 "moderator": true,102 "admin": true,103 "staff": true,104 "user_id": 211,105 "hidden": false,106 "trust_level": 4,107 "deleted_at": null,108 "user_deleted": false,109 "edit_reason": null,110 "can_view_edit_history": true,111 "wiki": false,112 "post_url": "/t/how-to-fix-when-pytorch-randomizing-sampling-keeps-yielding-nan/194678/2",113 "can_accept_answer": false,114 "can_unaccept_answer": false,115 "accepted_answer": false,116 "topic_accepted_answer": null117 },118 {119 "id": 429048,120 "name": "Martin Nguyen",121 "username": "Martin_Nguyen",122 "avatar_template": "/user_avatar/discuss.pytorch.org/martin_nguyen/{size}/61717_2.png",123 "created_at": "2024-01-03T22:50:27.653Z",124 "cooked": "<p>Thank you <a class=\"mention\" href=\"/u/alband\">@albanD</a>. I did trace back to the source, it happened before I fed the data into forward step. The process is like this:</p>\n<ul>\n<li>The sampling line of code (one of 4 trials I listed) was used to have random tensors.</li>\n<li>These random tensors were then fed into forward step (for weight and bias).</li>\n</ul>\n<p>Simply speaking: Sampling tensors for weight and bias using one of 4 trials I listed (<code>nan</code> appeared here) → some process → F.linear(x, w, b)</p>",125 "post_number": 5,126 "post_type": 1,127 "posts_count": 5,128 "updated_at": "2024-01-04T04:11:08.971Z",129 "reply_count": 1,130 "reply_to_post_number": 2,131 "quote_count": 0,132 "incoming_link_count": 0,133 "reads": 7,134 "readers_count": 6,135 "score": 6.4,136 "yours": false,137 "topic_id": 194678,138 "topic_slug": "how-to-fix-when-pytorch-randomizing-sampling-keeps-yielding-nan",139 "display_username": "Martin Nguyen",140 "primary_group_name": null,141 "flair_name": null,142 "flair_url": null,143 "flair_bg_color": null,144 "flair_color": null,145 "flair_group_id": null,146 "badges_granted": [],147 "version": 3,148 "can_edit": false,149 "can_delete": false,150 "can_recover": false,151 "can_see_hidden_post": false,152 "can_wiki": false,153 "read": true,154 "user_title": null,155 "reply_to_user": {156 "id": 211,157 "username": "albanD",158 "name": "Alban D",159 "avatar_template": "/user_avatar/discuss.pytorch.org/alband/{size}/215_2.png"160 },161 "bookmarked": false,162 "actions_summary": [],163 "moderator": false,164 "admin": false,165 "staff": false,166 "user_id": 72161,167 "hidden": false,168 "trust_level": 0,169 "deleted_at": null,170 "user_deleted": false,171 "edit_reason": null,172 "can_view_edit_history": true,173 "wiki": false,174 "post_url": "/t/how-to-fix-when-pytorch-randomizing-sampling-keeps-yielding-nan/194678/5",175 "can_accept_answer": false,176 "can_unaccept_answer": false,177 "accepted_answer": false,178 "topic_accepted_answer": null179 },180 {181 "id": 429082,182 "name": "Alban D",183 "username": "albanD",184 "avatar_template": "/user_avatar/discuss.pytorch.org/alband/{size}/215_2.png",185 "created_at": "2024-01-04T10:51:39.478Z",186 "cooked": "<p>This is quite surprising, running <code>torch.empty(120).uniform_(-.01, 0.01)</code> again and again locally never generates <code>nan</code>.</p>",187 "post_number": 6,188 "post_type": 1,189 "posts_count": 5,190 "updated_at": "2024-01-04T10:51:39.478Z",191 "reply_count": 1,192 "reply_to_post_number": 5,193 "quote_count": 0,194 "incoming_link_count": 0,195 "reads": 7,196 "readers_count": 6,197 "score": 6.4,198 "yours": false,199 "topic_id": 194678,200 "topic_slug": "how-to-fix-when-pytorch-randomizing-sampling-keeps-yielding-nan",201 "display_username": "Alban D",202 "primary_group_name": null,203 "flair_name": null,204 "flair_url": null,205 "flair_bg_color": null,206 "flair_color": null,207 "flair_group_id": null,208 "badges_granted": [],209 "version": 1,210 "can_edit": false,211 "can_delete": false,212 "can_recover": false,213 "can_see_hidden_post": false,214 "can_wiki": false,215 "read": true,216 "user_title": "",217 "reply_to_user": {218 "id": 72161,219 "username": "Martin_Nguyen",220 "name": "Martin Nguyen",221 "avatar_template": "/user_avatar/discuss.pytorch.org/martin_nguyen/{size}/61717_2.png"222 },223 "bookmarked": false,224 "actions_summary": [],225 "moderator": true,226 "admin": true,227 "staff": true,228 "user_id": 211,229 "hidden": false,230 "trust_level": 4,231 "deleted_at": null,232 "user_deleted": false,233 "edit_reason": null,234 "can_view_edit_history": true,235 "wiki": false,236 "post_url": "/t/how-to-fix-when-pytorch-randomizing-sampling-keeps-yielding-nan/194678/6",237 "can_accept_answer": false,238 "can_unaccept_answer": false,239 "accepted_answer": false,240 "topic_accepted_answer": null241 },242 {243 "id": 429582,244 "name": "Martin Nguyen",245 "username": "Martin_Nguyen",246 "avatar_template": "/user_avatar/discuss.pytorch.org/martin_nguyen/{size}/61717_2.png",247 "created_at": "2024-01-09T00:11:38.010Z",248 "cooked": "<p>Thanks <a class=\"mention\" href=\"/u/alband\">@albanD</a>. Yea unfortunately, it cannot be reproduced easily like that. I have been trying to see if I can reproduce it. Otherwise, I will keep changing “set_seed” until there is a number of “set_seed” that does not yield out any <code>nan</code>.</p>",249 "post_number": 7,250 "post_type": 1,251 "posts_count": 5,252 "updated_at": "2024-01-09T00:11:38.010Z",253 "reply_count": 0,254 "reply_to_post_number": 6,255 "quote_count": 0,256 "incoming_link_count": 0,257 "reads": 6,258 "readers_count": 5,259 "score": 1.2,260 "yours": false,261 "topic_id": 194678,262 "topic_slug": "how-to-fix-when-pytorch-randomizing-sampling-keeps-yielding-nan",263 "display_username": "Martin Nguyen",264 "primary_group_name": null,265 "flair_name": null,266 "flair_url": null,267 "flair_bg_color": null,268 "flair_color": null,269 "flair_group_id": null,270 "badges_granted": [],271 "version": 1,272 "can_edit": false,273 "can_delete": false,274 "can_recover": false,275 "can_see_hidden_post": false,276 "can_wiki": false,277 "read": true,278 "user_title": null,279 "reply_to_user": {280 "id": 211,281 "username": "albanD",282 "name": "Alban D",283 "avatar_template": "/user_avatar/discuss.pytorch.org/alband/{size}/215_2.png"284 },285 "bookmarked": false,286 "actions_summary": [],287 "moderator": false,288 "admin": false,289 "staff": false,290 "user_id": 72161,291 "hidden": false,292 "trust_level": 0,293 "deleted_at": null,294 "user_deleted": false,295 "edit_reason": null,296 "can_view_edit_history": true,297 "wiki": false,298 "post_url": "/t/how-to-fix-when-pytorch-randomizing-sampling-keeps-yielding-nan/194678/7",299 "can_accept_answer": false,300 "can_unaccept_answer": false,301 "accepted_answer": false,302 "topic_accepted_answer": null303 }304 ],305 "stream": [306 428831,307 428973,308 429048,309 429082,310 429582311 ]312 },313 "timeline_lookup": [314 [315 1,316 663317 ],318 [319 2,320 661321 ],322 [323 4,324 660325 ],326 [327 5,328 656329 ]330 ],331 "suggested_topics": [332 {333 "fancy_title": "How to deal with when some of the input is None?",334 "id": 214383,335 "title": "How to deal with when some of the input is None?",336 "slug": "how-to-deal-with-when-some-of-the-input-is-none",337 "posts_count": 8,338 "reply_count": 5,339 "highest_post_number": 9,340 "image_url": null,341 "created_at": "2024-12-19T02:23:42.499Z",342 "last_posted_at": "2025-01-03T00:06:45.685Z",343 "bumped": true,344 "bumped_at": "2025-01-03T00:06:45.685Z",345 "archetype": "regular",346 "unseen": false,347 "pinned": false,348 "unpinned": null,349 "visible": true,350 "closed": false,351 "archived": false,352 "bookmarked": null,353 "liked": null,354 "tags_descriptions": {},355 "like_count": 0,356 "views": 158,357 "category_id": 1,358 "featured_link": null,359 "has_accepted_answer": false,360 "posters": [361 {362 "extras": "latest",363 "description": "Original Poster, Most Recent Poster",364 "user": {365 "id": 81580,366 "username": "moondeokjong",367 "name": "덕종 문",368 "avatar_template": "/user_avatar/discuss.pytorch.org/moondeokjong/{size}/74604_2.png",369 "trust_level": 1370 }371 },372 {373 "extras": null,374 "description": "Frequent Poster",375 "user": {376 "id": 3534,377 "username": "ptrblck",378 "name": "",379 "avatar_template": "/user_avatar/discuss.pytorch.org/ptrblck/{size}/1823_2.png",380 "admin": true,381 "moderator": true,382 "trust_level": 2383 }384 },385 {386 "extras": null,387 "description": "Frequent Poster",388 "user": {389 "id": 81673,390 "username": "m_rf",391 "name": "m rf",392 "avatar_template": "/user_avatar/discuss.pytorch.org/m_rf/{size}/74689_2.png",393 "trust_level": 1394 }395 }396 ]397 },398 {399 "fancy_title": "Inconsistencies between PyTorch and NumPy when performing 32-bit floating-point sums",400 "id": 212931,401 "title": "Inconsistencies between PyTorch and NumPy when performing 32-bit floating-point sums",402 "slug": "inconsistencies-between-pytorch-and-numpy-when-performing-32-bit-floating-point-sums",403 "posts_count": 5,404 "reply_count": 3,405 "highest_post_number": 5,406 "image_url": null,407 "created_at": "2024-11-13T14:10:23.178Z",408 "last_posted_at": "2024-11-13T19:58:41.733Z",409 "bumped": true,410 "bumped_at": "2024-11-13T19:58:41.733Z",411 "archetype": "regular",412 "unseen": false,413 "pinned": false,414 "unpinned": null,415 "visible": true,416 "closed": false,417 "archived": false,418 "bookmarked": null,419 "liked": null,420 "tags_descriptions": {},421 "like_count": 2,422 "views": 122,423 "category_id": 1,424 "featured_link": null,425 "has_accepted_answer": true,426 "posters": [427 {428 "extras": "latest",429 "description": "Original Poster, Most Recent Poster",430 "user": {431 "id": 80884,432 "username": "sgolodetz",433 "name": "Stuart Golodetz",434 "avatar_template": "/user_avatar/discuss.pytorch.org/sgolodetz/{size}/73975_2.png",435 "trust_level": 1436 }437 },438 {439 "extras": null,440 "description": "Frequent Poster, Accepted Answer",441 "user": {442 "id": 3534,443 "username": "ptrblck",444 "name": "",445 "avatar_template": "/user_avatar/discuss.pytorch.org/ptrblck/{size}/1823_2.png",446 "admin": true,447 "moderator": true,448 "trust_level": 2449 }450 }451 ]452 },453 {454 "fancy_title": "Q: Trying to run Mandelbrot on GPU via PyTorch, but not seeing speed-up",455 "id": 213528,456 "title": "Q: Trying to run Mandelbrot on GPU via PyTorch, but not seeing speed-up",457 "slug": "q-trying-to-run-mandelbrot-on-gpu-via-pytorch-but-not-seeing-speed-up",458 "posts_count": 5,459 "reply_count": 2,460 "highest_post_number": 5,461 "image_url": "https://discuss.pytorch.org/uploads/default/optimized/3X/b/b/bb7f1190813ce66ca53039c469cf71a315b39d5e_2_1024x673.jpeg",462 "created_at": "2024-11-27T14:37:49.259Z",463 "last_posted_at": "2024-11-30T14:39:48.153Z",464 "bumped": true,465 "bumped_at": "2024-11-30T14:39:48.153Z",466 "archetype": "regular",467 "unseen": false,468 "pinned": false,469 "unpinned": null,470 "visible": true,471 "closed": false,472 "archived": false,473 "bookmarked": null,474 "liked": null,475 "tags_descriptions": {},476 "like_count": 0,477 "views": 306,478 "category_id": 1,479 "featured_link": null,480 "has_accepted_answer": true,481 "posters": [482 {483 "extras": "latest",484 "description": "Original Poster, Most Recent Poster",485 "user": {486 "id": 81164,487 "username": "Rajeev_Raizada",488 "name": "Rajeev Raizada",489 "avatar_template": "/user_avatar/discuss.pytorch.org/rajeev_raizada/{size}/74226_2.png",490 "trust_level": 0491 }492 },493 {494 "extras": null,495 "description": "Frequent Poster, Accepted Answer",496 "user": {497 "id": 3534,498 "username": "ptrblck",499 "name": "",500 "avatar_template": "/user_avatar/discuss.pytorch.org/ptrblck/{size}/1823_2.png",501 "admin": true,502 "moderator": true,503 "trust_level": 2504 }505 },506 {507 "extras": null,508 "description": "Frequent Poster",509 "user": {510 "id": 81166,511 "username": "lennarteingunia",512 "name": "Lennart Eing",513 "avatar_template": "/letter_avatar_proxy/v4/letter/l/cdc98d/{size}.png",514 "trust_level": 0515 }516 }517 ]518 },519 {520 "fancy_title": "Error while implementing BYOL Training Strategy",521 "id": 212422,522 "title": "Error while implementing BYOL Training Strategy",523 "slug": "error-while-implementing-byol-training-strategy",524 "posts_count": 4,525 "reply_count": 2,526 "highest_post_number": 4,527 "image_url": null,528 "created_at": "2024-11-01T18:13:24.323Z",529 "last_posted_at": "2024-11-02T18:19:56.396Z",530 "bumped": true,531 "bumped_at": "2024-11-02T18:19:56.396Z",532 "archetype": "regular",533 "unseen": false,534 "pinned": false,535 "unpinned": null,536 "visible": true,537 "closed": false,538 "archived": false,539 "bookmarked": null,540 "liked": null,541 "tags_descriptions": {},542 "like_count": 0,543 "views": 64,544 "category_id": 1,545 "featured_link": null,546 "has_accepted_answer": true,547 "posters": [548 {549 "extras": null,550 "description": "Original Poster",551 "user": {552 "id": 80558,553 "username": "Rebantadey",554 "name": "",555 "avatar_template": "/letter_avatar_proxy/v4/letter/r/6f9a4e/{size}.png",556 "trust_level": 1557 }558 },559 {560 "extras": "latest",561 "description": "Most Recent Poster, Accepted Answer",562 "user": {563 "id": 3534,564 "username": "ptrblck",565 "name": "",566 "avatar_template": "/user_avatar/discuss.pytorch.org/ptrblck/{size}/1823_2.png",567 "admin": true,568 "moderator": true,569 "trust_level": 2570 }571 }572 ]573 },574 {575 "fancy_title": "Is there a vectorize map function for tensor with different shape",576 "id": 214777,577 "title": "Is there a vectorize map function for tensor with different shape",578 "slug": "is-there-a-vectorize-map-function-for-tensor-with-different-shape",579 "posts_count": 3,580 "reply_count": 0,581 "highest_post_number": 3,582 "image_url": null,583 "created_at": "2024-12-30T07:12:48.347Z",584 "last_posted_at": "2024-12-31T03:05:48.254Z",585 "bumped": true,586 "bumped_at": "2024-12-31T03:05:48.254Z",587 "archetype": "regular",588 "unseen": false,589 "pinned": false,590 "unpinned": null,591 "visible": true,592 "closed": false,593 "archived": false,594 "bookmarked": null,595 "liked": null,596 "tags_descriptions": {},597 "like_count": 3,598 "views": 77,599 "category_id": 1,600 "featured_link": null,601 "has_accepted_answer": false,602 "posters": [603 {604 "extras": null,605 "description": "Original Poster",606 "user": {607 "id": 72471,608 "username": "shadowshadow",609 "name": "",610 "avatar_template": "/user_avatar/discuss.pytorch.org/shadowshadow/{size}/62985_2.png",611 "trust_level": 2612 }613 },614 {615 "extras": null,616 "description": "Frequent Poster",617 "user": {618 "id": 3534,619 "username": "ptrblck",620 "name": "",621 "avatar_template": "/user_avatar/discuss.pytorch.org/ptrblck/{size}/1823_2.png",622 "admin": true,623 "moderator": true,624 "trust_level": 2625 }626 },627 {628 "extras": "latest",629 "description": "Most Recent Poster",630 "user": {631 "id": 41396,632 "username": "soulitzer",633 "name": "",634 "avatar_template": "/letter_avatar_proxy/v4/letter/s/839c29/{size}.png",635 "trust_level": 2636 }637 }638 ]639 }640 ],641 "tags_descriptions": {},642 "fancy_title": "How to fix when pytorch randomizing/sampling keeps yielding nan?",643 "id": 194678,644 "title": "How to fix when pytorch randomizing/sampling keeps yielding nan?",645 "posts_count": 5,646 "created_at": "2024-01-02T01:30:10.852Z",647 "views": 379,648 "reply_count": 4,649 "like_count": 0,650 "last_posted_at": "2024-01-09T00:11:38.010Z",651 "visible": true,652 "closed": false,653 "archived": false,654 "has_summary": false,655 "archetype": "regular",656 "slug": "how-to-fix-when-pytorch-randomizing-sampling-keeps-yielding-nan",657 "category_id": 1,658 "word_count": 671,659 "deleted_at": null,660 "user_id": 72161,661 "featured_link": null,662 "pinned_globally": false,663 "pinned_at": null,664 "pinned_until": null,665 "image_url": "https://discuss.pytorch.org/uploads/default/original/3X/a/6/a679cdf2aa7ca749d997c509a9dc221568bfa1ad.png",666 "slow_mode_seconds": 0,667 "draft": null,668 "draft_key": "topic_194678",669 "draft_sequence": null,670 "unpinned": null,671 "pinned": false,672 "current_post_number": 1,673 "highest_post_number": 7,674 "deleted_by": null,675 "actions_summary": [676 {677 "id": 4,678 "count": 0,679 "hidden": false,680 "can_act": false681 },682 {683 "id": 8,684 "count": 0,685 "hidden": false,686 "can_act": false687 },688 {689 "id": 10,690 "count": 0,691 "hidden": false,692 "can_act": false693 },694 {695 "id": 7,696 "count": 0,697 "hidden": false,698 "can_act": false699 }700 ],701 "chunk_size": 20,702 "bookmarked": false,703 "topic_timer": null,704 "message_bus_last_id": 0,705 "participant_count": 2,706 "show_read_indicator": false,707 "thumbnails": [708 {709 "max_width": null,710 "max_height": null,711 "width": 528,712 "height": 99,713 "url": "https://discuss.pytorch.org/uploads/default/original/3X/a/6/a679cdf2aa7ca749d997c509a9dc221568bfa1ad.png"714 }715 ],716 "slow_mode_enabled_until": null,717 "can_vote": false,718 "vote_count": 0,719 "user_voted": false,720 "discourse_zendesk_plugin_zendesk_id": null,721 "discourse_zendesk_plugin_zendesk_url": "https://your-url.zendesk.com/agent/tickets/",722 "details": {723 "can_edit": false,724 "notification_level": 1,725 "participants": [726 {727 "id": 72161,728 "username": "Martin_Nguyen",729 "name": "Martin Nguyen",730 "avatar_template": "/user_avatar/discuss.pytorch.org/martin_nguyen/{size}/61717_2.png",731 "post_count": 3,732 "primary_group_name": null,733 "flair_name": null,734 "flair_url": null,735 "flair_color": null,736 "flair_bg_color": null,737 "flair_group_id": null,738 "trust_level": 0739 },740 {741 "id": 211,742 "username": "albanD",743 "name": "Alban D",744 "avatar_template": "/user_avatar/discuss.pytorch.org/alband/{size}/215_2.png",745 "post_count": 2,746 "primary_group_name": null,747 "flair_name": null,748 "flair_url": null,749 "flair_color": null,750 "flair_bg_color": null,751 "flair_group_id": null,752 "admin": true,753 "moderator": true,754 "trust_level": 4755 }756 ],757 "created_by": {758 "id": 72161,759 "username": "Martin_Nguyen",760 "name": "Martin Nguyen",761 "avatar_template": "/user_avatar/discuss.pytorch.org/martin_nguyen/{size}/61717_2.png"762 },763 "last_poster": {764 "id": 72161,765 "username": "Martin_Nguyen",766 "name": "Martin Nguyen",767 "avatar_template": "/user_avatar/discuss.pytorch.org/martin_nguyen/{size}/61717_2.png"768 }769 },770 "bookmarks": []771 },772 {773 "post_stream": {774 "posts": [775 {776 "id": 159169,777 "name": "Tomer P",778 "username": "TomerP",779 "avatar_template": "/letter_avatar_proxy/v4/letter/t/bc79bd/{size}.png",780 "created_at": "2020-01-16T17:44:22.920Z",781 "cooked": "<p>Hi, I’ve been following the tutorial for <a href=\"https://pytorch.org/tutorials/intermediate/tensorboard_tutorial.html\" rel=\"nofollow noopener\">Visalizing models, data, and training with Tensorboard</a> and got stuck at section 4 of the tutorial - Adding a “Projector” to TensorBoard.</p>\n<p>After I run the code and refresh the Tensorboard page at <a href=\"http://localhost:6006/\" rel=\"nofollow noopener\">http://localhost:6006/</a> , a Projector tab doesn’t appear. When I choose Projector from the drop down menu on the right side, I get an empty page. Then, in the cmd window I get the following error:</p>\n<pre><code class=\"lang-auto\">TensorFlow installation not found - running with reduced feature set.\nServing TensorBoard on localhost; to expose to the network, use a proxy or pass --bind_all\nTensorBoard 2.0.0 at http://localhost:6006/ (Press CTRL+C to quit)\nException in thread ProjectorPluginIsActiveThread:\nTraceback (most recent call last):\n File \"C:\\Users\\theto\\Miniconda3\\envs\\pytorch\\lib\\threading.py\", line 926, in _bootstrap_inner\n self.run()\n File \"C:\\Users\\theto\\Miniconda3\\envs\\pytorch\\lib\\threading.py\", line 870, in run\n self._target(*self._args, **self._kwargs)\n File \"C:\\Users\\theto\\Miniconda3\\envs\\pytorch\\lib\\site-packages\\tensorboard\\plugins\\projector\\projector_plugin.py\", line 322, in _determine_is_active\n if self.configs:\n File \"C:\\Users\\theto\\Miniconda3\\envs\\pytorch\\lib\\site-packages\\tensorboard\\plugins\\projector\\projector_plugin.py\", line 340, in configs\n self._augment_configs_with_checkpoint_info()\n File \"C:\\Users\\theto\\Miniconda3\\envs\\pytorch\\lib\\site-packages\\tensorboard\\plugins\\projector\\projector_plugin.py\", line 362, in _augment_configs_with_checkpoint_info\n tensor = _read_tensor_tsv_file(fpath)\n File \"C:\\Users\\theto\\Miniconda3\\envs\\pytorch\\lib\\site-packages\\tensorboard\\plugins\\projector\\projector_plugin.py\", line 157, in _read_tensor_tsv_file\n return np.array(tensor, dtype='float32')\nValueError: setting an array element with a sequence.\n</code></pre>\n<p>I’m running Pytorch 1.3.1 with Tensorboard 2.0.0 in a conda environment.</p>\n<p>Any idea why this is happening?</p>",782 "post_number": 1,783 "post_type": 1,784 "posts_count": 6,785 "updated_at": "2020-01-16T17:44:22.920Z",786 "reply_count": 0,787 "reply_to_post_number": null,788 "quote_count": 0,789 "incoming_link_count": 762,790 "reads": 26,791 "readers_count": 25,792 "score": 3820.2,793 "yours": false,794 "topic_id": 66888,795 "topic_slug": "projector-tab-doesnt-appear",796 "display_username": "Tomer P",797 "primary_group_name": null,798 "flair_name": null,799 "flair_url": null,800 "flair_bg_color": null,801 "flair_color": null,802 "flair_group_id": null,803 "badges_granted": [],804 "version": 1,805 "can_edit": false,806 "can_delete": false,807 "can_recover": false,808 "can_see_hidden_post": false,809 "can_wiki": false,810 "link_counts": [811 {812 "url": "https://pytorch.org/tutorials/intermediate/tensorboard_tutorial.html",813 "internal": false,814 "reflection": false,815 "title": "Visualizing Models, Data, and Training with TensorBoard — PyTorch Tutorials 1.3.1 documentation",816 "clicks": 7817 },818 {819 "url": "http://localhost:6006/",820 "internal": false,821 "reflection": false,822 "clicks": 0823 }824 ],825 "read": true,826 "user_title": null,827 "bookmarked": false,828 "actions_summary": [829 {830 "id": 2,831 "count": 1832 }833 ],834 "moderator": false,835 "admin": false,836 "staff": false,837 "user_id": 13631,838 "hidden": false,839 "trust_level": 1,840 "deleted_at": null,841 "user_deleted": false,842 "edit_reason": null,843 "can_view_edit_history": true,844 "wiki": false,845 "post_url": "/t/projector-tab-doesnt-appear/66888/1",846 "can_accept_answer": false,847 "can_unaccept_answer": false,848 "accepted_answer": false,849 "topic_accepted_answer": true,850 "can_vote": false851 },852 {853 "id": 179168,854 "name": "firing_neuron",855 "username": "Rafay_Khan",856 "avatar_template": "/user_avatar/discuss.pytorch.org/rafay_khan/{size}/22389_2.png",857 "created_at": "2020-04-04T15:14:37.454Z",858 "cooked": "<p>Same here the Projector Tab does not appear, but I can choose it from the drop down menu and seems to be working from there.</p>",859 "post_number": 2,860 "post_type": 1,861 "posts_count": 6,862 "updated_at": "2020-04-04T15:14:37.454Z",863 "reply_count": 0,864 "reply_to_post_number": null,865 "quote_count": 0,866 "incoming_link_count": 3,867 "reads": 19,868 "readers_count": 18,869 "score": 18.8,870 "yours": false,871 "topic_id": 66888,872 "topic_slug": "projector-tab-doesnt-appear",873 "display_username": "firing_neuron",874 "primary_group_name": null,875 "flair_name": null,876 "flair_url": null,877 "flair_bg_color": null,878 "flair_color": null,879 "flair_group_id": null,880 "badges_granted": [],881 "version": 1,882 "can_edit": false,883 "can_delete": false,884 "can_recover": false,885 "can_see_hidden_post": false,886 "can_wiki": false,887 "read": true,888 "user_title": "",889 "bookmarked": false,890 "actions_summary": [],891 "moderator": false,892 "admin": false,893 "staff": false,894 "user_id": 27139,895 "hidden": false,896 "trust_level": 1,897 "deleted_at": null,898 "user_deleted": false,899 "edit_reason": null,900 "can_view_edit_history": true,901 "wiki": false,902 "post_url": "/t/projector-tab-doesnt-appear/66888/2",903 "can_accept_answer": false,904 "can_unaccept_answer": false,905 "accepted_answer": false,906 "topic_accepted_answer": true907 },908 {909 "id": 182780,910 "name": "Aladdin Persson",911 "username": "AladdinPerzon",912 "avatar_template": "/user_avatar/discuss.pytorch.org/aladdinperzon/{size}/13045_2.png",913 "created_at": "2020-04-15T19:22:19.816Z",914 "cooked": "<p>I got the same problem, anyone managed to solve this?</p>\n<p>EDIT: Using Pytorch 1.4.0 and installing tensorboard nightly (2.3.0a I believe I installed) then it worked perfectly for me. Might be worth checking if you run into the same issue.</p>",915 "post_number": 3,916 "post_type": 1,917 "posts_count": 6,918 "updated_at": "2020-04-24T14:56:23.220Z",919 "reply_count": 0,920 "reply_to_post_number": null,921 "quote_count": 0,922 "incoming_link_count": 9,923 "reads": 19,924 "readers_count": 18,925 "score": 63.8,926 "yours": false,927 "topic_id": 66888,928 "topic_slug": "projector-tab-doesnt-appear",929 "display_username": "Aladdin Persson",930 "primary_group_name": null,931 "flair_name": null,932 "flair_url": null,933 "flair_bg_color": null,934 "flair_color": null,935 "flair_group_id": null,936 "badges_granted": [],937 "version": 2,938 "can_edit": false,939 "can_delete": false,940 "can_recover": false,941 "can_see_hidden_post": false,942 "can_wiki": false,943 "read": true,944 "user_title": null,945 "bookmarked": false,946 "actions_summary": [947 {948 "id": 2,949 "count": 1950 }951 ],952 "moderator": false,953 "admin": false,954 "staff": false,955 "user_id": 10565,956 "hidden": false,957 "trust_level": 2,958 "deleted_at": null,959 "user_deleted": false,960 "edit_reason": null,961 "can_view_edit_history": true,962 "wiki": false,963 "post_url": "/t/projector-tab-doesnt-appear/66888/3",964 "can_accept_answer": false,965 "can_unaccept_answer": false,966 "accepted_answer": false,967 "topic_accepted_answer": true968 },969 {970 "id": 188389,971 "name": "波 波",972 "username": "MrLi",973 "avatar_template": "/user_avatar/discuss.pytorch.org/mrli/{size}/23636_2.png",974 "created_at": "2020-05-02T07:35:55.380Z",975 "cooked": "<p>I have this problem too.<br>\nMy solution is to install tensorflow as backend support.</p>",976 "post_number": 4,977 "post_type": 1,978 "posts_count": 6,979 "updated_at": "2020-05-02T07:35:55.380Z",980 "reply_count": 0,981 "reply_to_post_number": null,982 "quote_count": 0,983 "incoming_link_count": 6,984 "reads": 16,985 "readers_count": 15,986 "score": 33.2,987 "yours": false,988 "topic_id": 66888,989 "topic_slug": "projector-tab-doesnt-appear",990 "display_username": "波 波",991 "primary_group_name": null,992 "flair_name": null,993 "flair_url": null,994 "flair_bg_color": null,995 "flair_color": null,996 "flair_group_id": null,997 "badges_granted": [],998 "version": 1,999 "can_edit": false,1000 "can_delete": false,1001 "can_recover": false,1002 "can_see_hidden_post": false,1003 "can_wiki": false,1004 "read": true,1005 "user_title": null,1006 "bookmarked": false,1007 "actions_summary": [],1008 "moderator": false,1009 "admin": false,1010 "staff": false,1011 "user_id": 30956,1012 "hidden": false,1013 "trust_level": 0,1014 "deleted_at": null,1015 "user_deleted": false,1016 "edit_reason": null,1017 "can_view_edit_history": true,1018 "wiki": false,1019 "post_url": "/t/projector-tab-doesnt-appear/66888/4",1020 "can_accept_answer": false,1021 "can_unaccept_answer": false,1022 "accepted_answer": false,1023 "topic_accepted_answer": true1024 },1025 {1026 "id": 192180,1027 "name": "Neo4dev",1028 "username": "neo4dev",1029 "avatar_template": "/letter_avatar_proxy/v4/letter/n/2bfe46/{size}.png",1030 "created_at": "2020-05-13T04:11:25.500Z",1031 "cooked": "<p>I had this problem as well, and the solution is to restart tensorboard in the terminal</p>",1032 "post_number": 5,1033 "post_type": 1,1034 "posts_count": 6,1035 "updated_at": "2020-05-13T04:11:25.500Z",1036 "reply_count": 0,1037 "reply_to_post_number": null,1038 "quote_count": 0,1039 "incoming_link_count": 6,1040 "reads": 15,1041 "readers_count": 14,1042 "score": 33.0,1043 "yours": false,1044 "topic_id": 66888,1045 "topic_slug": "projector-tab-doesnt-appear",1046 "display_username": "Neo4dev",1047 "primary_group_name": null,1048 "flair_name": null,1049 "flair_url": null,1050 "flair_bg_color": null,1051 "flair_color": null,1052 "flair_group_id": null,1053 "badges_granted": [],1054 "version": 1,1055 "can_edit": false,1056 "can_delete": false,1057 "can_recover": false,1058 "can_see_hidden_post": false,1059 "can_wiki": false,1060 "read": true,1061 "user_title": null,1062 "bookmarked": false,1063 "actions_summary": [],1064 "moderator": false,1065 "admin": false,1066 "staff": false,1067 "user_id": 31528,1068 "hidden": false,1069 "trust_level": 0,1070 "deleted_at": null,1071 "user_deleted": false,1072 "edit_reason": null,1073 "can_view_edit_history": true,1074 "wiki": false,1075 "post_url": "/t/projector-tab-doesnt-appear/66888/5",1076 "can_accept_answer": false,1077 "can_unaccept_answer": false,1078 "accepted_answer": false,1079 "topic_accepted_answer": true1080 },1081 {1082 "id": 429579,1083 "name": "r vn alst",1084 "username": "r_vn_alst",1085 "avatar_template": "/user_avatar/discuss.pytorch.org/r_vn_alst/{size}/66824_2.png",1086 "created_at": "2024-01-08T21:11:38.467Z",1087 "cooked": "<p>I got this problem (no projector tab to be seen) too, in my case I noticed that on the dropdown (upper right) there was a tab profile, that complained I should “pip install -U tensorboard-plugin-profile”.</p>\n<p>after doing that, there were a whole bunch of extra options (hparams, mesh, whatiftool) in the dropdown, and also projector.<br>\nHope it helps someone !</p>",1088 "post_number": 6,1089 "post_type": 1,1090 "posts_count": 6,1091 "updated_at": "2024-01-08T21:13:00.490Z",1092 "reply_count": 0,1093 "reply_to_post_number": null,1094 "quote_count": 0,1095 "incoming_link_count": 3,1096 "reads": 5,1097 "readers_count": 4,1098 "score": 16.0,1099 "yours": false,1100 "topic_id": 66888,1101 "topic_slug": "projector-tab-doesnt-appear",1102 "display_username": "r vn alst",1103 "primary_group_name": null,1104 "flair_name": null,1105 "flair_url": null,1106 "flair_bg_color": null,1107 "flair_color": null,1108 "flair_group_id": null,1109 "badges_granted": [],1110 "version": 1,1111 "can_edit": false,1112 "can_delete": false,1113 "can_recover": false,1114 "can_see_hidden_post": false,1115 "can_wiki": false,1116 "read": true,1117 "user_title": null,1118 "bookmarked": false,1119 "actions_summary": [],1120 "moderator": false,1121 "admin": false,1122 "staff": false,1123 "user_id": 72342,1124 "hidden": false,1125 "trust_level": 0,1126 "deleted_at": null,1127 "user_deleted": false,1128 "edit_reason": null,1129 "can_view_edit_history": true,1130 "wiki": false,1131 "post_url": "/t/projector-tab-doesnt-appear/66888/6",1132 "can_accept_answer": false,1133 "can_unaccept_answer": false,1134 "accepted_answer": true,1135 "topic_accepted_answer": true1136 }1137 ],1138 "stream": [1139 159169,1140 179168,1141 182780,1142 188389,1143 192180,1144 4295791145 ]1146 },1147 "timeline_lookup": [1148 [1149 1,1150 21091151 ],1152 [1153 2,1154 20301155 ],1156 [1157 3,1158 20191159 ],1160 [1161 4,1162 20021163 ],1164 [1165 5,1166 19921167 ],1168 [1169 6,1170 6561171 ]1172 ],1173 "suggested_topics": [1174 {1175 "fancy_title": "Build from source failing",1176 "id": 217994,1177 "title": "Build from source failing",1178 "slug": "build-from-source-failing",1179 "posts_count": 5,1180 "reply_count": 2,1181 "highest_post_number": 5,1182 "image_url": null,1183 "created_at": "2025-03-18T16:16:04.698Z",1184 "last_posted_at": "2025-09-10T06:11:17.025Z",1185 "bumped": true,1186 "bumped_at": "2025-09-10T06:11:17.025Z",1187 "archetype": "regular",1188 "unseen": false,1189 "pinned": false,1190 "unpinned": null,1191 "visible": true,1192 "closed": false,1193 "archived": false,1194 "bookmarked": null,1195 "liked": null,1196 "tags_descriptions": {},1197 "like_count": 0,1198 "views": 211,1199 "category_id": 1,1200 "featured_link": null,