DIYer22 · GitHub

Reproduction BUG code

import torch
from torch.autograd import Variable
x = Variable(torch.FloatTensor([1.,1]), requires_grad=True)
div = Variable(torch.FloatTensor([0.,1]))
y = x/div # => y is [inf, 1]
zero_mask = (div==0) # => zero_mask is [1, 0]
y[zero_mask] = 0  # => y is [0, 1]
loss = y.sum()
loss.backward()
print(x.grad) # grad is [nan, 1], but expected [0, 1]

Computational graph of loss not include x[0]
So, gradient of x[0] should be 0, but get NaN

more simple reproduction

x = Variable(torch.FloatTensor([1.,1]), requires_grad=True)
div = Variable(torch.FloatTensor([0.,1]))
y = x/div # => y is [inf, 1]
mask = (div!=0) # => mask is [0, 1]
loss = y[mask]
loss.backward()
print(x.grad) # grad is [nan, 1], but expected [0, 1]

Versions:

  • Python: 2.7
  • pyTorch: 0.3.0.post4

cc @svekars @holly1238 @ezyang @albanD @zou3519 @gqchen @pearu @nikitaved @soulitzer @lezcano @Varal7 @brianjo @mruberry @gchanan @bdhirsh @jbschlosser @anjali411 @jlin27

Read the original on github.com ↗