Anurag1734/cuda-error-resolution-analysis
07
1[2 {3 "post_stream": {4 "posts": [5 {6 "id": 387795,7 "name": "",8 "username": "abc50111",9 "avatar_template": "/letter_avatar_proxy/v4/letter/a/0ea827/{size}.png",10 "created_at": "2023-02-15T17:18:01.839Z",11 "cooked": "<p>I made a composite model <code>MainModel</code> which consist of a <code>GinEncoder</code> and a <code>MainModel</code> which containing some <code>Linear</code> layers, and the <code>GinEncoder</code> made by the package <code>torch-geometric</code>, show as following codes :</p>\n<pre><code class=\"lang-python\">class GinEncoder(torch.nn.Module):\n def __init__(self):\n super(GinEncoder, self).__init__()\n self.gin_convs = torch.nn.ModuleList()\n self.gin_convs.append(GINConv(Sequential(Linear(1, 4),\n BatchNorm1d(4), ReLU(),\n Linear(4, 4), ReLU())))\n self.gin_convs.append(GINConv(Sequential(Linear(4, 4),\n BatchNorm1d(4), ReLU(),\n Linear(4, 4), ReLU())))\n\n\n def forward(self, x, edge_index, batch_node_id):\n # Node embeddings\n nodes_emb_layers = []\n for i in range(2):\n x = self.gin_convs[i](x, edge_index)\n nodes_emb_layers.append(x)\n\n # Graph-level readout\n nodes_emb_pools = [global_add_pool(nodes_emb, batch_node_id) for nodes_emb in nodes_emb_layers]\n\n # Concatenate and form the graph embeddings\n graph_embeds = torch.cat(nodes_emb_pools, dim=1)\n return graph_embeds\n\n\n def get_embeddings(self, x, edge_index, batch_node_id):\n with torch.no_grad():\n graph_embeds = self.forward(x, edge_index, batch_node_id).reshape(-1)\n\n return graph_embeds\n\n\nclass MainModel(torch.nn.Module):\n def __init__(self, graph_encoder:torch.nn.Module):\n super(MainModel, self).__init__()\n self.graph_encoder = graph_encoder\n self.lin1 = Linear(8, 4)\n self.lin2 = Linear(4, 8)\n\n\n def forward(self, x, edge_index, batch_node_id):\n graph_embeds = self.graph_encoder(x, edge_index, batch_node_id)\n out_lin1 = self.lin1(graph_embeds)\n pred = self.lin2(out_lin1)[-1]\n\n return pred\n\ngin_encoder = GinEncoder().to(\"cuda\")\nmodel = MainModel(gin_encoder).to(\"cuda\")\n</code></pre>\n<p>I found that the weights of <code>GinEncoder</code> were not updated, while the weights of <code>Linear</code> layer in <code>MainModel</code> were updated.I observe this by following codes:</p>\n<pre><code class=\"lang-python\">gin_encoder = GinEncoder().to(\"cuda\")\nmodel = MainModel(gin_encoder).to(\"cuda\")\ncriterion = torch.nn.MSELoss()\noptimizer = torch.optim.Adam(model.parameters())\nepochs = \n\nfor epoch_i in range(epochs):\n model.train()\n train_loss = 0\n\n for batch_i, data in enumerate(train_loader):\n data.to(\"cuda\")\n x, x_edge_index, x_batch_node_id = data.x, data.edge_index, data.batch\n y, y_edge_index, y_batch_node_id = data.y[-1].x, data.y[-1].edge_index, torch.zeros(data.y[-1].x.shape[0], dtype=torch.int64).to(\"cuda\")\n optimizer.zero_grad()\n graph_embeds_pred = model(x, x_edge_index, x_batch_node_id)\n y_graph_embeds = model.graph_encoder.get_embeddings(y, y_edge_index, y_batch_node_id)\n loss = criterion(graph_embeds_pred, y_graph_embeds)\n train_loss += loss\n loss.backward()\n optimizer.step()\n if batch_i == 0:\n print(f\"NO. {epoch_i} EPOCH\")\n print(f\"MainModel weights in epoch_{epoch_i}_batch0:{next(islice(model.parameters(), 15, 16))}\", end=\"\\n\\n\")\n print(f\"GinEncoder weights in epoch_{epoch_i}_batch0:{next(model.graph_encoder.parameters())}\")\n print(\"*\"*80)\n</code></pre>\n<p>Outputs of codes:</p>\n<pre><code class=\"lang-auto\">NO. 0 EPOCH\nMainModel weights in epoch_0_batch0:Parameter containing:\ntensor([-0.1447, -0.3689, -0.2840, -0.3619, -0.2040, 0.2430, 0.4651, 0.3736],\n device='cuda:0', requires_grad=True)\n\nGinEncoder weights in epoch_0_batch0:Parameter containing:\ntensor([[-0.8312],\n [-0.5712],\n [-0.6963],\n [-0.1601]], device='cuda:0', requires_grad=True)\n********************************************************************************\nNO. 1 EPOCH\nMainModel weights in epoch_1_batch0:Parameter containing:\ntensor([-0.1842, -0.3333, -0.3170, -0.3247, -0.2424, 0.2627, 0.4272, 0.4119],\n device='cuda:0', requires_grad=True)\n\nGinEncoder weights in epoch_1_batch0:Parameter containing:\ntensor([[-0.8312],\n [-0.5712],\n [-0.6963],\n [-0.1601]], device='cuda:0', requires_grad=True)\n********************************************************************************\nNO. 2 EPOCH\nMainModel weights in epoch_2_batch0:Parameter containing:\ntensor([-0.2302, -0.3077, -0.3251, -0.2905, -0.2847, 0.2558, 0.3881, 0.4527],\n device='cuda:0', requires_grad=True)\n\nGinEncoder weights in epoch_2_batch0:Parameter containing:\ntensor([[-0.8312],\n [-0.5712],\n [-0.6963],\n [-0.1601]], device='cuda:0', requires_grad=True)\n********************************************************************************\n</code></pre>\n<p>My question is how to make <code>loss.backward()</code> and <code>optimizer.step()</code> also pass to <code>GinEncoder</code>?</p>\n<p>PS.</p>\n<ul>\n<li>I put the complete codes in here: <a href=\"https://gist.github.com/theabc50111/3ca708d0c1101d57b6172bd717302710\" class=\"inline-onebox\" rel=\"noopener nofollow ugc\">a composite model composed of pytorch and torch-geometric · GitHub</a>\n</li>\n<li>I put the training data on Google Drive: <a href=\"https://drive.google.com/drive/folders/1_KMwCzf1diwS4gGNdSSxG7bnemqQkFxI?usp=sharing\" class=\"inline-onebox\" rel=\"noopener nofollow ugc\">tmp - Google Drive</a>\n</li>\n</ul>",12 "post_number": 1,13 "post_type": 1,14 "posts_count": 6,15 "updated_at": "2023-02-15T17:18:01.839Z",16 "reply_count": 0,17 "reply_to_post_number": null,18 "quote_count": 0,19 "incoming_link_count": 432,20 "reads": 8,21 "readers_count": 7,22 "score": 2161.6,23 "yours": false,24 "topic_id": 172658,25 "topic_slug": "pytorch-geometric-gin-conv-layers-parameters-not-updating",26 "display_username": "",27 "primary_group_name": null,28 "flair_name": null,29 "flair_url": null,30 "flair_bg_color": null,31 "flair_color": null,32 "flair_group_id": null,33 "badges_granted": [],34 "version": 1,35 "can_edit": false,36 "can_delete": false,37 "can_recover": false,38 "can_see_hidden_post": false,39 "can_wiki": false,40 "link_counts": [41 {42 "url": "https://gist.github.com/theabc50111/3ca708d0c1101d57b6172bd717302710",43 "internal": false,44 "reflection": false,45 "title": "a composite model composed of pytorch and torch-geometric · GitHub",46 "clicks": 147 },48 {49 "url": "https://drive.google.com/drive/folders/1_KMwCzf1diwS4gGNdSSxG7bnemqQkFxI?usp=sharing",50 "internal": false,51 "reflection": false,52 "title": "tmp - Google Drive",53 "clicks": 054 }55 ],56 "read": true,57 "user_title": null,58 "bookmarked": false,59 "actions_summary": [],60 "moderator": false,61 "admin": false,62 "staff": false,63 "user_id": 63386,64 "hidden": false,65 "trust_level": 1,66 "deleted_at": null,67 "user_deleted": false,68 "edit_reason": null,69 "can_view_edit_history": true,70 "wiki": false,71 "post_url": "/t/pytorch-geometric-gin-conv-layers-parameters-not-updating/172658/1",72 "can_accept_answer": false,73 "can_unaccept_answer": false,74 "accepted_answer": false,75 "topic_accepted_answer": null,76 "can_vote": false77 },78 {79 "id": 387893,80 "name": "",81 "username": "ptrblck",82 "avatar_template": "/user_avatar/discuss.pytorch.org/ptrblck/{size}/1823_2.png",83 "created_at": "2023-02-16T07:44:01.241Z",84 "cooked": "<p>Could you check the <code>.grad</code> attribute of all parameters of the <code>GinEncoder</code> before and after the first <code>.backward</code> call to see if these gradients are calculated but might be small?</p>",85 "post_number": 2,86 "post_type": 1,87 "posts_count": 6,88 "updated_at": "2023-02-16T07:44:01.241Z",89 "reply_count": 1,90 "reply_to_post_number": null,91 "quote_count": 0,92 "incoming_link_count": 3,93 "reads": 4,94 "readers_count": 3,95 "score": 20.8,96 "yours": false,97 "topic_id": 172658,98 "topic_slug": "pytorch-geometric-gin-conv-layers-parameters-not-updating",99 "display_username": "",100 "primary_group_name": null,101 "flair_name": null,102 "flair_url": null,103 "flair_bg_color": null,104 "flair_color": null,105 "flair_group_id": null,106 "badges_granted": [],107 "version": 1,108 "can_edit": false,109 "can_delete": false,110 "can_recover": false,111 "can_see_hidden_post": false,112 "can_wiki": false,113 "read": true,114 "user_title": "",115 "bookmarked": false,116 "actions_summary": [],117 "moderator": true,118 "admin": true,119 "staff": true,120 "user_id": 3534,121 "hidden": false,122 "trust_level": 2,123 "deleted_at": null,124 "user_deleted": false,125 "edit_reason": null,126 "can_view_edit_history": true,127 "wiki": false,128 "post_url": "/t/pytorch-geometric-gin-conv-layers-parameters-not-updating/172658/2",129 "can_accept_answer": false,130 "can_unaccept_answer": false,131 "accepted_answer": false,132 "topic_accepted_answer": null133 },134 {135 "id": 388294,136 "name": "",137 "username": "abc50111",138 "avatar_template": "/letter_avatar_proxy/v4/letter/a/0ea827/{size}.png",139 "created_at": "2023-02-18T09:14:38.187Z",140 "cooked": "<p>Thank you for your patience in reading my question.</p>\n<p>I observe that the <strong>gradient of parameters Gin Model is always 0</strong>.</p>\n<p>I tried to use following codes to observe the <code>.grad</code> attribute of <em>the first layer of parameters of the <code>GinEncoder</code></em>:</p>\n<pre><code class=\"lang-auto\"> x, x_edge_index, x_batch_node_id = data.x, data.edge_index, data.batch\n y, y_edge_index, y_batch_node_id = data.y[-1].x, data.y[-1].edge_index, torch.zeros(data.y[-1].x.shape[0], dtype=torch.int64).to(\"cuda\")\n model_optimizer.zero_grad()\n graph_embeds_pred = model(x, x_edge_index, x_batch_node_id)\n y_graph_embeds = model.graph_encoder.get_embeddings(y, y_edge_index, y_batch_node_id)\n loss = criterion(graph_embeds_pred, y_graph_embeds)\n train_loss += loss\n print(f\"Before loss.backward(), MainModel weights.grad in epoch_{epoch_i}_batch{batch_i}:{next(islice(model.parameters(), 15, 16)).grad}\", end=\"\\n\\n\")\n print(f\"Before loss.backward(), MainModel.graph_encoder weights.grad in epoch_{epoch_i}_batch{batch_i}:{next(model.graph_encoder.parameters()).grad}\")\n loss.backward()\n print(f\"After loss.backward(), MainModel weights.grad in epoch_{epoch_i}_batch{batch_i}:{next(islice(model.parameters(), 15, 16)).grad}\", end=\"\\n\\n\")\n print(f\"After loss.backward(), MainModel.graph_encoder weights.grad in epoch_{epoch_i}_batch{batch_i}:{next(model.graph_encoder.parameters()).grad}\")\n print(\"*\"*80)\n</code></pre>\n<p>The result in epoch 0 & <strong>batch 0</strong>:</p>\n<pre><code class=\"lang-auto\">Before loss.backward(), MainModel weights.grad in epoch_0_batch0:None\n\nBefore loss.backward(), MainModel.graph_encoder weights.grad in epoch_0_batch0:None\nAfter loss.backward(), MainModel weights.grad in epoch_0_batch0:tensor([-0.0839, 0.0596, -0.1096, 0.0718, 0.1150, 0.0749, 0.0800, 0.0076],\n device='cuda:0')\n\nAfter loss.backward(), MainModel.graph_encoder weights.grad in epoch_0_batch0:tensor([[0.],\n [0.],\n [0.],\n [0.]], device='cuda:0')\n</code></pre>\n<p>The result in epoch 0 & <strong>batch 1</strong>:</p>\n<pre><code class=\"lang-auto\">Before loss.backward(), MainModel weights.grad in epoch_0_batch1:tensor([0., 0., 0., 0., 0., 0., 0., 0.], device='cuda:0')\n\nBefore loss.backward(), MainModel.graph_encoder weights.grad in epoch_0_batch1:tensor([[0.],\n [0.],\n [0.],\n [0.]], device='cuda:0')\nAfter loss.backward(), MainModel weights.grad in epoch_0_batch1:tensor([-0.0640, 0.0315, -0.0785, 0.0666, 0.1209, 0.0641, 0.0495, -0.0090],\n device='cuda:0')\n\nAfter loss.backward(), MainModel.graph_encoder weights.grad in epoch_0_batch1:tensor([[0.],\n [0.],\n [0.],\n [0.]], device='cuda:0')\n</code></pre>",141 "post_number": 3,142 "post_type": 1,143 "posts_count": 6,144 "updated_at": "2023-02-18T09:14:38.187Z",145 "reply_count": 1,146 "reply_to_post_number": 2,147 "quote_count": 0,148 "incoming_link_count": 2,149 "reads": 4,150 "readers_count": 3,151 "score": 15.8,152 "yours": false,153 "topic_id": 172658,154 "topic_slug": "pytorch-geometric-gin-conv-layers-parameters-not-updating",155 "display_username": "",156 "primary_group_name": null,157 "flair_name": null,158 "flair_url": null,159 "flair_bg_color": null,160 "flair_color": null,161 "flair_group_id": null,162 "badges_granted": [],163 "version": 1,164 "can_edit": false,165 "can_delete": false,166 "can_recover": false,167 "can_see_hidden_post": false,168 "can_wiki": false,169 "read": true,170 "user_title": null,171 "reply_to_user": {172 "id": 3534,173 "username": "ptrblck",174 "name": "",175 "avatar_template": "/user_avatar/discuss.pytorch.org/ptrblck/{size}/1823_2.png"176 },177 "bookmarked": false,178 "actions_summary": [],179 "moderator": false,180 "admin": false,181 "staff": false,182 "user_id": 63386,183 "hidden": false,184 "trust_level": 1,185 "deleted_at": null,186 "user_deleted": false,187 "edit_reason": null,188 "can_view_edit_history": true,189 "wiki": false,190 "post_url": "/t/pytorch-geometric-gin-conv-layers-parameters-not-updating/172658/3",191 "can_accept_answer": false,192 "can_unaccept_answer": false,193 "accepted_answer": false,194 "topic_accepted_answer": null195 },196 {197 "id": 388300,198 "name": "",199 "username": "ptrblck",200 "avatar_template": "/user_avatar/discuss.pytorch.org/ptrblck/{size}/1823_2.png",201 "created_at": "2023-02-18T10:05:24.584Z",202 "cooked": "<p>OK, the results at least show that you are not detaching the operations from the computation graph since the <code>.grad</code> attributes are at least populated.<br>\nI don’t know enough about the <code>GINConv</code> implementation to comment why the gradients might be zero, but <a class=\"mention\" href=\"/u/rusty1s\">@rusty1s</a> might know.</p>",203 "post_number": 4,204 "post_type": 1,205 "posts_count": 6,206 "updated_at": "2023-02-18T10:05:24.584Z",207 "reply_count": 1,208 "reply_to_post_number": 3,209 "quote_count": 0,210 "incoming_link_count": 0,211 "reads": 4,212 "readers_count": 3,213 "score": 5.8,214 "yours": false,215 "topic_id": 172658,216 "topic_slug": "pytorch-geometric-gin-conv-layers-parameters-not-updating",217 "display_username": "",218 "primary_group_name": null,219 "flair_name": null,220 "flair_url": null,221 "flair_bg_color": null,222 "flair_color": null,223 "flair_group_id": null,224 "badges_granted": [],225 "version": 1,226 "can_edit": false,227 "can_delete": false,228 "can_recover": false,229 "can_see_hidden_post": false,230 "can_wiki": false,231 "read": true,232 "user_title": "",233 "reply_to_user": {234 "id": 63386,235 "username": "abc50111",236 "name": "",237 "avatar_template": "/letter_avatar_proxy/v4/letter/a/0ea827/{size}.png"238 },239 "bookmarked": false,240 "actions_summary": [],241 "moderator": true,242 "admin": true,243 "staff": true,244 "user_id": 3534,245 "hidden": false,246 "trust_level": 2,247 "deleted_at": null,248 "user_deleted": false,249 "edit_reason": null,250 "can_view_edit_history": true,251 "wiki": false,252 "post_url": "/t/pytorch-geometric-gin-conv-layers-parameters-not-updating/172658/4",253 "can_accept_answer": false,254 "can_unaccept_answer": false,255 "accepted_answer": false,256 "topic_accepted_answer": null257 },258 {259 "id": 388330,260 "name": "",261 "username": "abc50111",262 "avatar_template": "/letter_avatar_proxy/v4/letter/a/0ea827/{size}.png",263 "created_at": "2023-02-18T14:35:36.301Z",264 "cooked": "<p>Thank You for reply.</p>\n<p>I have tried to merge the two class: <code>MainModel</code> and <code>GinEncoder()</code> to avoid the unchanging parameters of <code>gin_covs()</code>.</p>\n<p>It works, it made the parameters of <code>gin_convs()</code> change, <strong>but this approach is still not answer my confusion</strong></p>\n<p>Here is the update codes:</p>\n<pre><code class=\"lang-auto\">class MainModel2(torch.nn.Module):\n def __init__(self):\n super(MainModel2, self).__init__()\n self.gin_convs = torch.nn.ModuleList()\n self.gin_convs.append(GINConv(Sequential(Linear(1, 4), ReLU(),\n Linear(4, 4), ReLU(),\n BatchNorm1d(4))))\n self.gin_convs.append(GINConv(Sequential(Linear(4, 4), ReLU(),\n Linear(4, 4), ReLU(),\n BatchNorm1d(4))))\n self.lin1 = Linear(8, 4)\n self.lin2 = Linear(4, 8)\n\n\n def forward(self, x, edge_index, batch_node_id):\n # Node embeddings\n nodes_emb_layers = []\n for i in range(2):\n x = self.gin_convs[i](x, edge_index)\n nodes_emb_layers.append(x)\n\n # Graph-level readout\n nodes_emb_pools = [global_add_pool(nodes_emb, batch_node_id) for nodes_emb in nodes_emb_layers]\n\n # Concatenate and form the graph embeddings\n graph_embeds = torch.cat(nodes_emb_pools, dim=1)\n out_lin1 = self.lin1(graph_embeds)\n pred = self.lin2(out_lin1)[-1]\n\n return pred\n</code></pre>",265 "post_number": 5,266 "post_type": 1,267 "posts_count": 6,268 "updated_at": "2023-02-18T14:35:36.301Z",269 "reply_count": 0,270 "reply_to_post_number": 4,271 "quote_count": 0,272 "incoming_link_count": 3,273 "reads": 5,274 "readers_count": 4,275 "score": 16.0,276 "yours": false,277 "topic_id": 172658,278 "topic_slug": "pytorch-geometric-gin-conv-layers-parameters-not-updating",279 "display_username": "",280 "primary_group_name": null,281 "flair_name": null,282 "flair_url": null,283 "flair_bg_color": null,284 "flair_color": null,285 "flair_group_id": null,286 "badges_granted": [],287 "version": 1,288 "can_edit": false,289 "can_delete": false,290 "can_recover": false,291 "can_see_hidden_post": false,292 "can_wiki": false,293 "read": true,294 "user_title": null,295 "reply_to_user": {296 "id": 3534,297 "username": "ptrblck",298 "name": "",299 "avatar_template": "/user_avatar/discuss.pytorch.org/ptrblck/{size}/1823_2.png"300 },301 "bookmarked": false,302 "actions_summary": [],303 "moderator": false,304 "admin": false,305 "staff": false,306 "user_id": 63386,307 "hidden": false,308 "trust_level": 1,309 "deleted_at": null,310 "user_deleted": false,311 "edit_reason": null,312 "can_view_edit_history": true,313 "wiki": false,314 "post_url": "/t/pytorch-geometric-gin-conv-layers-parameters-not-updating/172658/5",315 "can_accept_answer": false,316 "can_unaccept_answer": false,317 "accepted_answer": false,318 "topic_accepted_answer": null319 },320 {321 "id": 411465,322 "name": "",323 "username": "abc50111",324 "avatar_template": "/letter_avatar_proxy/v4/letter/a/0ea827/{size}.png",325 "created_at": "2023-07-26T08:26:23.312Z",326 "cooked": "<p>Update:<br>\nThe the weights are actually updated, I sums up all the <code>.grad</code> of every layers of <code>GinEncoder</code>.</p>\n<p>I find that the <code>.grad</code> of the first layer of <code>GinEncoder</code> weights <strong>occasionally</strong> is zero, it has nothing to do with how you build the model from pytorch and torch_geometric.</p>\n<p>So like <a class=\"mention\" href=\"/u/ptrblck\">@ptrblck</a> said, after check the <code>.grad</code> attribute of <strong>all</strong> parameters, the weights are actually updated, I just missed it.</p>",327 "post_number": 6,328 "post_type": 1,329 "posts_count": 6,330 "updated_at": "2023-07-26T08:26:23.312Z",331 "reply_count": 0,332 "reply_to_post_number": null,333 "quote_count": 0,334 "incoming_link_count": 3,335 "reads": 2,336 "readers_count": 1,337 "score": 15.4,338 "yours": false,339 "topic_id": 172658,340 "topic_slug": "pytorch-geometric-gin-conv-layers-parameters-not-updating",341 "display_username": "",342 "primary_group_name": null,343 "flair_name": null,344 "flair_url": null,345 "flair_bg_color": null,346 "flair_color": null,347 "flair_group_id": null,348 "badges_granted": [],349 "version": 1,350 "can_edit": false,351 "can_delete": false,352 "can_recover": false,353 "can_see_hidden_post": false,354 "can_wiki": false,355 "read": true,356 "user_title": null,357 "bookmarked": false,358 "actions_summary": [],359 "moderator": false,360 "admin": false,361 "staff": false,362 "user_id": 63386,363 "hidden": false,364 "trust_level": 1,365 "deleted_at": null,366 "user_deleted": false,367 "edit_reason": null,368 "can_view_edit_history": true,369 "wiki": false,370 "post_url": "/t/pytorch-geometric-gin-conv-layers-parameters-not-updating/172658/6",371 "can_accept_answer": false,372 "can_unaccept_answer": false,373 "accepted_answer": false,374 "topic_accepted_answer": null375 }376 ],377 "stream": [378 387795,379 387893,380 388294,381 388300,382 388330,383 411465384 ]385 },386 "timeline_lookup": [387 [388 1,389 983390 ],391 [392 2,393 982394 ],395 [396 3,397 980398 ],399 [400 6,401 822402 ]403 ],404 "suggested_topics": [405 {406 "fancy_title": "Scraping pytorch forums data",407 "id": 214244,408 "title": "Scraping pytorch forums data",409 "slug": "scraping-pytorch-forums-data",410 "posts_count": 1,411 "reply_count": 0,412 "highest_post_number": 1,413 "image_url": null,414 "created_at": "2024-12-15T14:25:10.197Z",415 "last_posted_at": "2024-12-15T14:25:10.234Z",416 "bumped": true,417 "bumped_at": "2024-12-15T14:25:10.234Z",418 "archetype": "regular",419 "unseen": false,420 "pinned": false,421 "unpinned": null,422 "visible": true,423 "closed": false,424 "archived": false,425 "bookmarked": null,426 "liked": null,427 "tags_descriptions": {},428 "like_count": 0,429 "views": 117,430 "category_id": 1,431 "featured_link": null,432 "has_accepted_answer": false,433 "posters": [434 {435 "extras": "latest single",436 "description": "Original Poster, Most Recent Poster",437 "user": {438 "id": 72190,439 "username": "Sai1",440 "name": "Sai",441 "avatar_template": "/user_avatar/discuss.pytorch.org/sai1/{size}/62072_2.png",442 "trust_level": 1443 }444 }445 ]446 },447 {448 "fancy_title": "`torch.linalg.svd` uses `cudaMemcpyAsync` that syncs between host and device",449 "id": 213297,450 "title": "`torch.linalg.svd` uses `cudaMemcpyAsync` that syncs between host and device",451 "slug": "torch-linalg-svd-uses-cudamemcpyasync-that-syncs-between-host-and-device",452 "posts_count": 1,453 "reply_count": 0,454 "highest_post_number": 1,455 "image_url": null,456 "created_at": "2024-11-22T07:35:51.287Z",457 "last_posted_at": "2024-11-22T07:35:51.352Z",458 "bumped": true,459 "bumped_at": "2024-11-22T07:35:51.352Z",460 "archetype": "regular",461 "unseen": false,462 "pinned": false,463 "unpinned": null,464 "visible": true,465 "closed": false,466 "archived": false,467 "bookmarked": null,468 "liked": null,469 "tags_descriptions": {},470 "like_count": 0,471 "views": 64,472 "category_id": 1,473 "featured_link": null,474 "has_accepted_answer": false,475 "posters": [476 {477 "extras": "latest single",478 "description": "Original Poster, Most Recent Poster",479 "user": {480 "id": 14714,481 "username": "Rui_Wang",482 "name": "Rui Wang",483 "avatar_template": "/user_avatar/discuss.pytorch.org/rui_wang/{size}/37635_2.png",484 "trust_level": 1485 }486 }487 ]488 },489 {490 "fancy_title": "Convert ONNX to PyTorch: TypeError: Conv2d.__init__() missing 2 required positional arguments: ‘in_channels’ and ‘out_channels’",491 "id": 215169,492 "title": "Convert ONNX to PyTorch: TypeError: Conv2d.__init__() missing 2 required positional arguments: 'in_channels' and 'out_channels'",493 "slug": "convert-onnx-to-pytorch-typeerror-conv2d-init-missing-2-required-positional-arguments-in-channels-and-out-channels",494 "posts_count": 3,495 "reply_count": 1,496 "highest_post_number": 3,497 "image_url": null,498 "created_at": "2025-01-09T12:45:15.734Z",499 "last_posted_at": "2025-01-10T13:45:36.292Z",500 "bumped": true,501 "bumped_at": "2025-01-10T13:45:36.292Z",502 "archetype": "regular",503 "unseen": false,504 "pinned": false,505 "unpinned": null,506 "visible": true,507 "closed": false,508 "archived": false,509 "bookmarked": null,510 "liked": null,511 "tags_descriptions": {},512 "like_count": 0,513 "views": 92,514 "category_id": 1,515 "featured_link": null,516 "has_accepted_answer": false,517 "posters": [518 {519 "extras": "latest",520 "description": "Original Poster, Most Recent Poster",521 "user": {522 "id": 65147,523 "username": "natalia_meira",524 "name": "Natalia Meira",525 "avatar_template": "/user_avatar/discuss.pytorch.org/natalia_meira/{size}/59369_2.png",526 "trust_level": 1527 }528 },529 {530 "extras": null,531 "description": "Frequent Poster",532 "user": {533 "id": 3534,534 "username": "ptrblck",535 "name": "",536 "avatar_template": "/user_avatar/discuss.pytorch.org/ptrblck/{size}/1823_2.png",537 "admin": true,538 "moderator": true,539 "trust_level": 2540 }541 }542 ]543 },544 {545 "fancy_title": "Torch version for python-3.10.8",546 "id": 215511,547 "title": "Torch version for python-3.10.8",548 "slug": "torch-version-for-python-3-10-8",549 "posts_count": 3,550 "reply_count": 1,551 "highest_post_number": 3,552 "image_url": null,553 "created_at": "2025-01-17T08:32:52.449Z",554 "last_posted_at": "2025-01-17T16:20:54.686Z",555 "bumped": true,556 "bumped_at": "2025-01-17T16:20:54.686Z",557 "archetype": "regular",558 "unseen": false,559 "pinned": false,560 "unpinned": null,561 "visible": true,562 "closed": false,563 "archived": false,564 "bookmarked": null,565 "liked": null,566 "tags_descriptions": {},567 "like_count": 0,568 "views": 140,569 "category_id": 1,570 "featured_link": null,571 "has_accepted_answer": false,572 "posters": [573 {574 "extras": "latest",575 "description": "Original Poster, Most Recent Poster",576 "user": {577 "id": 82141,578 "username": "Frank_Liu",579 "name": "Frank Liu",580 "avatar_template": "/user_avatar/discuss.pytorch.org/frank_liu/{size}/73474_2.png",581 "trust_level": 0582 }583 },584 {585 "extras": null,586 "description": "Frequent Poster",587 "user": {588 "id": 3534,589 "username": "ptrblck",590 "name": "",591 "avatar_template": "/user_avatar/discuss.pytorch.org/ptrblck/{size}/1823_2.png",592 "admin": true,593 "moderator": true,594 "trust_level": 2595 }596 }597 ]598 },599 {600 "fancy_title": "Nan values caused by scatter",601 "id": 216931,602 "title": "Nan values caused by scatter",603 "slug": "nan-values-caused-by-scatter",604 "posts_count": 1,605 "reply_count": 0,606 "highest_post_number": 1,607 "image_url": null,608 "created_at": "2025-02-20T07:18:49.680Z",609 "last_posted_at": "2025-02-20T07:18:49.722Z",610 "bumped": true,611 "bumped_at": "2025-02-20T07:18:49.722Z",612 "archetype": "regular",613 "unseen": false,614 "pinned": false,615 "unpinned": null,616 "visible": true,617 "closed": false,618 "archived": false,619 "bookmarked": null,620 "liked": null,621 "tags_descriptions": {},622 "like_count": 0,623 "views": 36,624 "category_id": 1,625 "featured_link": null,626 "has_accepted_answer": false,627 "posters": [628 {629 "extras": "latest single",630 "description": "Original Poster, Most Recent Poster",631 "user": {632 "id": 82813,633 "username": "gitKincses",634 "name": "Git Kincses",635 "avatar_template": "/user_avatar/discuss.pytorch.org/gitkincses/{size}/75774_2.png",636 "trust_level": 0637 }638 }639 ]640 }641 ],642 "tags_descriptions": {},643 "fancy_title": "PyTorch Geometric GIN-Conv layers parameters not updating",644 "id": 172658,645 "title": "PyTorch Geometric GIN-Conv layers parameters not updating",646 "posts_count": 6,647 "created_at": "2023-02-15T17:18:01.659Z",648 "views": 870,649 "reply_count": 3,650 "like_count": 0,651 "last_posted_at": "2023-07-26T08:26:23.312Z",652 "visible": true,653 "closed": false,654 "archived": false,655 "has_summary": false,656 "archetype": "regular",657 "slug": "pytorch-geometric-gin-conv-layers-parameters-not-updating",658 "category_id": 1,659 "word_count": 1217,660 "deleted_at": null,661 "user_id": 63386,662 "featured_link": null,663 "pinned_globally": false,664 "pinned_at": null,665 "pinned_until": null,666 "image_url": null,667 "slow_mode_seconds": 0,668 "draft": null,669 "draft_key": "topic_172658",670 "draft_sequence": null,671 "unpinned": null,672 "pinned": false,673 "current_post_number": 1,674 "highest_post_number": 6,675 "deleted_by": null,676 "actions_summary": [677 {678 "id": 4,679 "count": 0,680 "hidden": false,681 "can_act": false682 },683 {684 "id": 8,685 "count": 0,686 "hidden": false,687 "can_act": false688 },689 {690 "id": 10,691 "count": 0,692 "hidden": false,693 "can_act": false694 },695 {696 "id": 7,697 "count": 0,698 "hidden": false,699 "can_act": false700 }701 ],702 "chunk_size": 20,703 "bookmarked": false,704 "topic_timer": null,705 "message_bus_last_id": 0,706 "participant_count": 2,707 "show_read_indicator": false,708 "thumbnails": null,709 "slow_mode_enabled_until": null,710 "can_vote": false,711 "vote_count": 0,712 "user_voted": false,713 "discourse_zendesk_plugin_zendesk_id": null,714 "discourse_zendesk_plugin_zendesk_url": "https://your-url.zendesk.com/agent/tickets/",715 "details": {716 "can_edit": false,717 "notification_level": 1,718 "participants": [719 {720 "id": 63386,721 "username": "abc50111",722 "name": "",723 "avatar_template": "/letter_avatar_proxy/v4/letter/a/0ea827/{size}.png",724 "post_count": 4,725 "primary_group_name": null,726 "flair_name": null,727 "flair_url": null,728 "flair_color": null,729 "flair_bg_color": null,730 "flair_group_id": null,731 "trust_level": 1732 },733 {734 "id": 3534,735 "username": "ptrblck",736 "name": "",737 "avatar_template": "/user_avatar/discuss.pytorch.org/ptrblck/{size}/1823_2.png",738 "post_count": 2,739 "primary_group_name": null,740 "flair_name": null,741 "flair_url": null,742 "flair_color": null,743 "flair_bg_color": null,744 "flair_group_id": null,745 "admin": true,746 "moderator": true,747 "trust_level": 2748 }749 ],750 "created_by": {751 "id": 63386,752 "username": "abc50111",753 "name": "",754 "avatar_template": "/letter_avatar_proxy/v4/letter/a/0ea827/{size}.png"755 },756 "last_poster": {757 "id": 63386,758 "username": "abc50111",759 "name": "",760 "avatar_template": "/letter_avatar_proxy/v4/letter/a/0ea827/{size}.png"761 },762 "links": [763 {764 "url": "https://gist.github.com/theabc50111/3ca708d0c1101d57b6172bd717302710",765 "title": "a composite model composed of pytorch and torch-geometric · GitHub",766 "internal": false,767 "attachment": false,768 "reflection": false,769 "clicks": 1,770 "user_id": 63386,771 "domain": "gist.github.com",772 "root_domain": "github.com"773 }774 ]775 },776 "bookmarks": []777 },778 {779 "post_stream": {780 "posts": [781 {782 "id": 411450,783 "name": "Jeong",784 "username": "ymin2570",785 "avatar_template": "/letter_avatar_proxy/v4/letter/y/a587f6/{size}.png",786 "created_at": "2023-07-26T07:23:12.318Z",787 "cooked": "<p>Hi, I am conducting on using layers of existing pretrained NN model for new NN model with additional module in existing NN model.</p>\n<p>The pretrained NN model is as follow:</p>\n<pre><code class=\"lang-python\">class DUNet(nn.Module):\n def __init__(self, in_ch, in_ch2, out_ch, out_ch2, bilinear):\n super(DUNet, self).__init__()\n self.encoder1 = UNetDown(in_ch=in_ch, bilinear=bilinear)\n self.encoder2 = UNetDown(in_ch=in_ch2, bilinear=bilinear)\n self.decoder1= UNetUp(out_ch=out_ch, bilinear=bilinear)\n self.decoder2= UNetUp(out_ch=out_ch2, bilinear=bilinear)\n self.outc = OutConv(4, 3)\n\n def forward(self, input_1, input_2):\n f1_1, f2_1, f3_1, f4_1, f5_1 = self.encoder1(input_1)\n f1_2, f2_2, f3_2, f4_2, f5_2 = self.encoder2(input_2)\n f6_1, f7_1, f8_1, f9_1, recon_1 = self.decoder1(f1_1, f2_1, f3_1, f4_1, f5_1)\n f6_2, f7_2, f8_2, f9_2, recon_2 = self.decoder2(f1_2, f2_2, f3_2, f4_2, f5_2)\n\n concat = torch.cat([recon_1, recon_2], dim=1)\n output = self.outc(concat)\n return output, f6_1, f7_1, f8_1, f9_1, f6_2, f7_2, f8_2, f9_2\n</code></pre>\n<p>it is two branch U-Net structure.<br>\nAnd new NN model is as follow:</p>\n<pre><code class=\"lang-python\">class DUNet_finetune(nn.Module):\n def __init__(self, in_ch, in_ch2, out_ch, out_ch2, bilinear):\n super(DUNet_finetune, self).__init__()\n self.pretrained_DUNet = DUNet(in_ch, in_ch2, out_ch, out_ch2, bilinear)\n self.encoder1 = self.pretrained_DUNet.encoder1\n self.encoder2 = self.pretrained_DUNet.encoder2\n ... #(override the other variables as well)\n\n # define additional module\n # def new_module():\n # ...\n\n def forward(self, input_1, input_2):\n f1_1, f2_1, f3_1, f4_1, f5_1 = self.encoder1(input_1)\n f1_2, f2_2, f3_2, f4_2, f5_2 = self.encoder2(input_2)\n f6_1, f7_1, f8_1, f9_1, recon_1 = self.decoder1(f1_1, f2_1, f3_1, f4_1, f5_1)\n f6_2, f7_2, f8_2, f9_2, recon_2 = self.decoder2(f1_2, f2_2, f3_2, f4_2, f5_2)\n # additional module operated in this section\n #new_module()\n #...\n\n concat = torch.cat([recon_1, recon_2], dim=1)\n output = self.outc(concat)\n return output, f6_1, f7_1, f8_1, f9_1, f6_2, f7_2, f8_2, f9_2\n</code></pre>\n<p>when I training <strong>DUNet</strong>, I wrap the model with nn.DataParallel. (Actually, It doesn’t needed but I didn’t changed.)<br>\nWhen I train <strong>DUNet_finetune</strong> model that using <strong>pretrained DUNet’s layer</strong>, the training code is as follow:</p>\n<pre><code class=\"lang-python\">net= DUNet_finetune(in_ch=3, in_ch2=1, out_ch=3, out_ch2=1, bilinear=False).cuda()\nnet= torch.nn.DataParallel(net)\n\nif opt.pretrained_guided:\n checkpoint = torch.load(PATH)\n net.module.pretrained_DUNet.load_state_dict(checkpoint['model_state_dict'])\n print('Use the Pretrained Network!')\n</code></pre>\n<p>And I get the error msg :<br>\n<strong>Error(s) in loading state_dict for Parallel:<br>\nMissing key(s) in state_dict: “encoder1.inc.conv_blocks.0.weight”, …<br>\nUnexpected key(s) in state_dict: “module.encoder1.inc.conv_blocks.0.weight”, …</strong></p>\n<p>I’ve solved this problem with</p>\n<pre><code class=\"lang-python\">net.module.pretrained_DUNet.load_state_dict(checkpoint['model_state_dict'], strict=False)\n</code></pre>\n<p>by referring to the contents shown <a href=\"https://discuss.pytorch.org/t/missing-keys-unexpected-keys-in-state-dict-when-loading-self-trained-model/22379/5\">here</a>.</p>\n<p>But I don’t know what this means. I am concerned about whether the network will be learned as I want. I want to finetune the DUNet with additional module.</p>\n<p>I don’t know if there is better solution (ex, train DUNet without wrapping nn.DataParallel or else).<br>\nCould you give me some advice?<br>\nThank you very much.</p>",788 "post_number": 1,789 "post_type": 1,790 "posts_count": 3,791 "updated_at": "2023-07-26T07:30:44.703Z",792 "reply_count": 1,793 "reply_to_post_number": null,794 "quote_count": 0,795 "incoming_link_count": 22,796 "reads": 6,797 "readers_count": 5,798 "score": 116.2,799 "yours": false,800 "topic_id": 185062,801 "topic_slug": "what-is-the-strict-false-factor-intended-for-model-load-state-dict",802 "display_username": "Jeong",803 "primary_group_name": null,804 "flair_name": null,805 "flair_url": null,806 "flair_bg_color": null,807 "flair_color": null,808 "flair_group_id": null,809 "badges_granted": [],810 "version": 2,811 "can_edit": false,812 "can_delete": false,813 "can_recover": false,814 "can_see_hidden_post": false,815 "can_wiki": false,816 "link_counts": [817 {818 "url": "https://discuss.pytorch.org/t/missing-keys-unexpected-keys-in-state-dict-when-loading-self-trained-model/22379/5",819 "internal": true,820 "reflection": false,821 "title": "Missing keys & unexpected keys in state_dict when loading self trained model",822 "clicks": 2823 }824 ],825 "read": true,826 "user_title": "",827 "bookmarked": false,828 "actions_summary": [],829 "moderator": false,830 "admin": false,831 "staff": false,832 "user_id": 66487,833 "hidden": false,834 "trust_level": 1,835 "deleted_at": null,836 "user_deleted": false,837 "edit_reason": null,838 "can_view_edit_history": true,839 "wiki": false,840 "post_url": "/t/what-is-the-strict-false-factor-intended-for-model-load-state-dict/185062/1",841 "can_accept_answer": false,842 "can_unaccept_answer": false,843 "accepted_answer": false,844 "topic_accepted_answer": null,845 "can_vote": false846 },847 {848 "id": 411454,849 "name": "",850 "username": "ptrblck",851 "avatar_template": "/user_avatar/discuss.pytorch.org/ptrblck/{size}/1823_2.png",852 "created_at": "2023-07-26T07:40:42.725Z",853 "cooked": "<aside class=\"quote no-group\" data-username=\"ymin2570\" data-post=\"1\" data-topic=\"185062\">\n<div class=\"title\">\n<div class=\"quote-controls\"></div>\n<img loading=\"lazy\" alt=\"\" width=\"24\" height=\"24\" src=\"https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ymin2570/48/60786_2.png\" class=\"avatar\"> ymin2570:</div>\n<blockquote>\n<p>I’ve solved this problem with</p>\n</blockquote>\n</aside>\n<p>You did not solve the issue, but you are explicitly ignoring the mismatches and most likely no parameters are loaded at all.<br>\n<code>strict=False</code> allows you to skip the mismatching keys and can be used for the linked use case, where the user added one additional module.<br>\nIn your case the <code>state_dict</code> contains the <code>.module</code> keys added by <code>nn.DataParallel</code> while you are trying to load it into the raw model inside the <code>nn.DataParallel</code> wrapper.<br>\nMake sure to store and load the same <code>state_dict</code>, ideally from the internal model (not from the <code>nn.DataParallel</code> model).</p>",854 "post_number": 2,855 "post_type": 1,856 "posts_count": 3,857 "updated_at": "2023-07-26T07:40:42.725Z",858 "reply_count": 1,859 "reply_to_post_number": null,860 "quote_count": 1,861 "incoming_link_count": 1,862 "reads": 4,863 "readers_count": 3,864 "score": 10.8,865 "yours": false,866 "topic_id": 185062,867 "topic_slug": "what-is-the-strict-false-factor-intended-for-model-load-state-dict",868 "display_username": "",869 "primary_group_name": null,870 "flair_name": null,871 "flair_url": null,872 "flair_bg_color": null,873 "flair_color": null,874 "flair_group_id": null,875 "badges_granted": [],876 "version": 1,877 "can_edit": false,878 "can_delete": false,879 "can_recover": false,880 "can_see_hidden_post": false,881 "can_wiki": false,882 "read": true,883 "user_title": "",884 "bookmarked": false,885 "actions_summary": [],886 "moderator": true,887 "admin": true,888 "staff": true,889 "user_id": 3534,890 "hidden": false,891 "trust_level": 2,892 "deleted_at": null,893 "user_deleted": false,894 "edit_reason": null,895 "can_view_edit_history": true,896 "wiki": false,897 "post_url": "/t/what-is-the-strict-false-factor-intended-for-model-load-state-dict/185062/2",898 "can_accept_answer": false,899 "can_unaccept_answer": false,900 "accepted_answer": false,901 "topic_accepted_answer": null902 },903 {904 "id": 411461,905 "name": "Jeong",906 "username": "ymin2570",907 "avatar_template": "/letter_avatar_proxy/v4/letter/y/a587f6/{size}.png",908 "created_at": "2023-07-26T08:01:07.015Z",909 "cooked": "<p>I understand you said “train DUNet without wrapping with nn.DataParallel.” I’ll give it a try. Thank you for your advice.</p>",910 "post_number": 3,911 "post_type": 1,912 "posts_count": 3,913 "updated_at": "2023-07-26T08:01:07.015Z",914 "reply_count": 0,915 "reply_to_post_number": 2,916 "quote_count": 0,917 "incoming_link_count": 2,918 "reads": 4,919 "readers_count": 3,920 "score": 10.8,921 "yours": false,922 "topic_id": 185062,923 "topic_slug": "what-is-the-strict-false-factor-intended-for-model-load-state-dict",924 "display_username": "Jeong",925 "primary_group_name": null,926 "flair_name": null,927 "flair_url": null,928 "flair_bg_color": null,929 "flair_color": null,930 "flair_group_id": null,931 "badges_granted": [],932 "version": 1,933 "can_edit": false,934 "can_delete": false,935 "can_recover": false,936 "can_see_hidden_post": false,937 "can_wiki": false,938 "read": true,939 "user_title": "",940 "reply_to_user": {941 "id": 3534,942 "username": "ptrblck",943 "name": "",944 "avatar_template": "/user_avatar/discuss.pytorch.org/ptrblck/{size}/1823_2.png"945 },946 "bookmarked": false,947 "actions_summary": [],948 "moderator": false,949 "admin": false,950 "staff": false,951 "user_id": 66487,952 "hidden": false,953 "trust_level": 1,954 "deleted_at": null,955 "user_deleted": false,956 "edit_reason": null,957 "can_view_edit_history": true,958 "wiki": false,959 "post_url": "/t/what-is-the-strict-false-factor-intended-for-model-load-state-dict/185062/3",960 "can_accept_answer": false,961 "can_unaccept_answer": false,962 "accepted_answer": false,963 "topic_accepted_answer": null964 }965 ],966 "stream": [967 411450,968 411454,969 411461970 ]971 },972 "timeline_lookup": [973 [974 1,975 822976 ]977 ],978 "suggested_topics": [979 {980 "fancy_title": "Good performance during evaluation, but poor performance during inference",981 "id": 212476,982 "title": "Good performance during evaluation, but poor performance during inference",983 "slug": "good-performance-during-evaluation-but-poor-performance-during-inference",984 "posts_count": 1,985 "reply_count": 0,986 "highest_post_number": 1,987 "image_url": null,988 "created_at": "2024-11-03T14:26:02.726Z",989 "last_posted_at": "2024-11-03T14:26:02.840Z",990 "bumped": true,991 "bumped_at": "2024-11-03T14:26:02.840Z",992 "archetype": "regular",993 "unseen": false,994 "pinned": false,995 "unpinned": null,996 "visible": true,997 "closed": false,998 "archived": false,999 "bookmarked": null,1000 "liked": null,1001 "tags_descriptions": {},1002 "like_count": 0,1003 "views": 32,1004 "category_id": 1,1005 "featured_link": null,1006 "has_accepted_answer": false,1007 "posters": [1008 {1009 "extras": "latest single",1010 "description": "Original Poster, Most Recent Poster",1011 "user": {1012 "id": 80666,1013 "username": "Rafi_Darmawan",1014 "name": "Rafi Darmawan",1015 "avatar_template": "/user_avatar/discuss.pytorch.org/rafi_darmawan/{size}/72921_2.png",1016 "trust_level": 01017 }1018 }1019 ]1020 },1021 {1022 "fancy_title": "Error when calulating recall",1023 "id": 215278,1024 "title": "Error when calulating recall",1025 "slug": "error-when-calulating-recall",1026 "posts_count": 6,1027 "reply_count": 3,1028 "highest_post_number": 6,1029 "image_url": null,1030 "created_at": "2025-01-11T21:37:01.913Z",1031 "last_posted_at": "2025-01-13T23:43:29.368Z",1032 "bumped": true,1033 "bumped_at": "2025-01-13T23:43:29.368Z",1034 "archetype": "regular",1035 "unseen": false,1036 "pinned": false,1037 "unpinned": null,1038 "visible": true,1039 "closed": false,1040 "archived": false,1041 "bookmarked": null,1042 "liked": null,1043 "tags_descriptions": {},1044 "like_count": 0,1045 "views": 263,1046 "category_id": 1,1047 "featured_link": null,1048 "has_accepted_answer": false,1049 "posters": [1050 {1051 "extras": null,1052 "description": "Original Poster",1053 "user": {1054 "id": 82027,1055 "username": "Joshua_Ossai",1056 "name": "Joshua Ossai",1057 "avatar_template": "/user_avatar/discuss.pytorch.org/joshua_ossai/{size}/75056_2.png",1058 "trust_level": 11059 }1060 },1061 {1062 "extras": "latest",1063 "description": "Most Recent Poster",1064 "user": {1065 "id": 3534,1066 "username": "ptrblck",1067 "name": "",1068 "avatar_template": "/user_avatar/discuss.pytorch.org/ptrblck/{size}/1823_2.png",1069 "admin": true,1070 "moderator": true,1071 "trust_level": 21072 }1073 }1074 ]1075 },1076 {1077 "fancy_title": "Optimizing difference of two outer products help",1078 "id": 214326,1079 "title": "Optimizing difference of two outer products help",1080 "slug": "optimizing-difference-of-two-outer-products-help",1081 "posts_count": 2,1082 "reply_count": 0,1083 "highest_post_number": 2,1084 "image_url": null,1085 "created_at": "2024-12-17T17:23:18.739Z",1086 "last_posted_at": "2024-12-17T21:31:49.025Z",1087 "bumped": true,1088 "bumped_at": "2024-12-17T21:31:49.025Z",1089 "archetype": "regular",1090 "unseen": false,1091 "pinned": false,1092 "unpinned": null,1093 "visible": true,1094 "closed": false,1095 "archived": false,1096 "bookmarked": null,1097 "liked": null,1098 "tags_descriptions": {},1099 "like_count": 1,1100 "views": 30,1101 "category_id": 1,1102 "featured_link": null,1103 "has_accepted_answer": false,1104 "posters": [1105 {1106 "extras": null,1107 "description": "Original Poster",1108 "user": {1109 "id": 57398,1110 "username": "Nyakov",1111 "name": "",1112 "avatar_template": "/user_avatar/discuss.pytorch.org/nyakov/{size}/51180_2.png",1113 "trust_level": 11114 }1115 },1116 {1117 "extras": "latest",1118 "description": "Most Recent Poster",1119 "user": {1120 "id": 41396,1121 "username": "soulitzer",1122 "name": "",1123 "avatar_template": "/letter_avatar_proxy/v4/letter/s/839c29/{size}.png",1124 "trust_level": 21125 }1126 }1127 ]1128 },1129 {1130 "fancy_title": "Nvidia N-body executing CUDA kernel with pytorch",1131 "id": 214635,1132 "title": "Nvidia N-body executing CUDA kernel with pytorch",1133 "slug": "nvidia-n-body-executing-cuda-kernel-with-pytorch",1134 "posts_count": 2,1135 "reply_count": 0,1136 "highest_post_number": 2,1137 "image_url": null,1138 "created_at": "2024-12-25T19:12:49.505Z",1139 "last_posted_at": "2024-12-25T23:25:25.282Z",1140 "bumped": true,1141 "bumped_at": "2024-12-25T23:25:25.282Z",1142 "archetype": "regular",1143 "unseen": false,1144 "pinned": false,1145 "unpinned": null,1146 "visible": true,1147 "closed": false,1148 "archived": false,1149 "bookmarked": null,1150 "liked": null,1151 "tags_descriptions": {},1152 "like_count": 0,1153 "views": 106,1154 "category_id": 1,1155 "featured_link": null,1156 "has_accepted_answer": false,1157 "posters": [1158 {1159 "extras": null,1160 "description": "Original Poster",1161 "user": {1162 "id": 69390,1163 "username": "Georges_Leukic",1164 "name": "Georges Leukic",1165 "avatar_template": "/user_avatar/discuss.pytorch.org/georges_leukic/{size}/63838_2.png",1166 "trust_level": 01167 }1168 },1169 {1170 "extras": "latest",1171 "description": "Most Recent Poster",1172 "user": {1173 "id": 3534,1174 "username": "ptrblck",1175 "name": "",1176 "avatar_template": "/user_avatar/discuss.pytorch.org/ptrblck/{size}/1823_2.png",1177 "admin": true,1178 "moderator": true,1179 "trust_level": 21180 }1181 }1182 ]1183 },1184 {1185 "fancy_title": "Use flexattention with torchrec",1186 "id": 215998,1187 "title": "Use flexattention with torchrec",1188 "slug": "use-flexattention-with-torchrec",1189 "posts_count": 2,1190 "reply_count": 0,1191 "highest_post_number": 2,1192 "image_url": null,1193 "created_at": "2025-01-28T17:55:21.621Z",1194 "last_posted_at": "2025-01-28T19:16:28.133Z",1195 "bumped": true,1196 "bumped_at": "2025-01-28T19:16:28.133Z",1197 "archetype": "regular",1198 "unseen": false,1199 "pinned": false,1200 "unpinned": null,