Question: tmpdir = tempfile.mkdtemp() ext_suf = '.pyd' if os.sys.platform == 'win32' else '.so' libname = cffi_wrapper_name + ext_suf <del> ffi.compile(tmpdir=tmpdir, verbose=verbose, target=libname) shutil.copy(os.path.join(tmpdir, libname), os.path.join(target_dir, libname)) </del> <ins> outfile = ffi.compile(tmpdir=tmpdir, verbose=verbose, target=libname) shutil.copy(outfile, os.path.join(target_dir, libname)) </ins> finally: shutil.rmtree(tmpdir)

(old title) When build C extension, the error: FileNotFoundError was got. OS: Windows 10 pro PyTorch version: 0.4.0a0+ How you installed PyTorch (conda, pip, source): source Python version: 3.6.4 CUDA/cuDNN version: CUDA 9.0 GCC version (if compiling from source): msvc 14/15 (then compiling with CUDA I use VS2015, but when build extension, the program automatically use vs2017) When building c extension on Windows, I got the error: (The Chinese above means compiler success compile the library, and generated .lib and .exp) And the same error was got on a Linux Work Station. (gcc is 5.4.0)
In , copy the linked file from , for example , but in my Windows, and a Linux work station, the linked one is in: ,for example . So, there must be a bug, or error in pytorch's ffi or python's ffi. (pyhton 3.6.4, cffi 1.11.4), If it broke down because of change in cffi, I think I can create a PR. I do think this is because of the change of cffi api,
CC who enabled extension build for Windows on
Is this change documented in somewhere like Python SDK?


Question: for(iw = 0; iw < kW; iw++) { real val = *(ip + it*istrideT + ih*istrideH + iw*istrideW); <del> if (val > maxval) </del> <ins> if ((val > maxval) || isnan(val)) </ins> { maxval = val; maxindex = (it+istartT)*isizeH*isizeW + (ih+istartH)*isizeW + (iw+istartW);

max pooling functions are not consistent with max functions. Below an example, every max pooling (be it 1d, 2d or 3d, adaptive or not) acts the same, on cpu or on cuda. Essentially, there are two fondamental differences : max pooling of all values is while for it's max pooling of nan and valid values is valid values, which means s get ignored, while for , as soon as there is a value, the result is . More generally, choosing explicetely how to deal with as in numpy () could be a solution, but maybe this is related to CuDNN's max pooling ? Built from latest sources (as of 05/17) PyTorch version: 0.5.0a0+ Is debug build: No CUDA used to build PyTorch: 9.1.85 OS: Ubuntu 16.04.4 LTS GCC version: (Ubuntu 5.4.0-6ubuntu1~16.04.9) 5.4.0 CMake version: version 3.9.4 Python version: 3.6 Is CUDA available: Yes CUDA runtime version: 9.1.85 GPU models and configuration: GPU 0: Quadro M1000M Nvidia driver version: 390.30 cuDNN version: Probably one of the following: /usr/lib/x8664-linux- /usr/lib/x8664-linux-gnu/libcudnnstaticv7.a Versions of relevant libraries: [conda] magma-cuda91 2.3.0 1 pytorch [conda] torch 0.5.0a0+ <pip[conda] torch 0.3.1b0+ <pip[conda] torch 0.5.0a0+ <pip
Hi, . Thank you for providing the example. I'll take a look.
Hmhm. I seem to get a double free in the ...
Another question: Do we really want to provide an option to ignore NaN? My view is that "if you have nan in your net, you're screwed", so I would just return NaN there, personally.
My practical usecase is for KITTI groundtruth depthmaps and FlowMaps which are 2D sparse arrays.


Question: for(iw = 0; iw < kW; iw++) { real val = *(ip + it*istrideT + ih*istrideH + iw*istrideW); <del> if (val > maxval) </del> <ins> if ((val > maxval) || isnan(val)) </ins> { maxval = val; maxindex = (it+istartT)*isizeH*isizeW + (ih+istartH)*isizeW + (iw+istartW);

For an algorithm that use FlowNet-like architecture that outputs predictions at multiple scale levels, we can either compare predictions to downscaled GT or upscaled predictions to GT. The first being obviously less computationally expensive, the ignore NaN would help downscaling such sparse 2D maps. For the moment we do something I find ugly, you can see it Essentially it zeroes the s, takes the map and construct two maps of positive and negative values which are then maxpooled and back together. I am actually open for a strict "no-nan" policy on pooling functions, but in that case better enforce it before someone writes a code that tries to benefit from maxpooling ignore s feature/bug and if you have a clever way of pooling sparse 2D tensors, I'm open to it, but I guess it's a topic for pytorch forums ;)
Personally, I think it is more sane have NaN -NaN in the pooling and offer a parametrizable (where you get to pick the values) that does or so.
So the proposed fix does NaN -NaN similar to max. I didn't try to fix gradients for the nan case. This would involve keeping the values and I don't think that is worth it (in particular because I would not expect the pooling layer to be last, and otherwise we'd probably get NaN as grad_out). If you are reasonably happy with it, I'd move it to a PR.
+1 for NaN -NaN, "abyssus abyssum invocat" We could also add an optional mask which would be a ByteTensor of the same size, specifying whether or not the considered pixel is used for the pooling, that could be used for any kind of pulling. An functionality would then be to provide the mask
I could work with that. to continue to be picky, the potential drawback is that it won't work for other pooling methods such as average pooling or median pooling. The problem here is that the max operation inherently ignores nonmax values which can be leveraged for "ignore some pixels" operation, but it has a "non universal" feel to it since it woun't be as easy for other kinds of 2D operations.
To be clear, the fix is good for me, but I figured a related discussion on selective pooling (whether to ditch NaN values or anything you want to ignore) could happen (maybe not on this issue ?)


Question: raise ValueError('num_workers cannot be negative; ' 'use num_workers=0 to disable multiprocessing.') <ins> if sys.platform == "win32" and self.num_workers > 0: raise ValueError('num_workers > 0 is not supported on Windows') </ins> if batch_sampler is None: if sampler is None: if shuffle:

This issue tracks the components / tests that are not working on Windows: [ ] : currently disabled on Windows because Windows doesn't have and we need to look for substitutes [ ] : Fuser is disabled on Windows because the current implementation uses symbols from Linux-specific headers and . We will need to find alternatives for Windows. [ ] : some parts of and are disabled because Windows doesn't support opening an already opened file (see discussion at and ) [ ] : currently doesn't work with Windows ( is the porting diff) [ ] : in causes intermittent CUDA out-of-memory error on Windows [ ] , : DataLoader with multiple workers causes intermittent CUDA out-of-memory error on Windows. [x] [x] (done in ) [x] [x] - - [x] [x] [x] - For more discussions, also see: cc
The first one is solved by but I think DataLoader can be further improved when is fininshed.
Cool I will mark it as resolved :)
The has been in .
We can try to revert and , because the memory leak in the CPU side could also cause CUDA errors.
Are they fixed by I think we can revert them after we merge the PR.
I think this may be related. Since once the memory of the CPU side is low, the will fail with too.
I think it could be. For what it's worth, when I tried to inspect the CUDA OOM error, showed no process that was taking memory, but running CUDA tests on the machine would still fail.
Now that is merged into master, could you please try to revert the changes on and ?
Awesome! Just to understand it better: does fix both numworker=1 and numworker1 cases?
Yes, they are both solved.
I guess we should mark the cpp_extension test as completed, for it's now enabled in CI.
Closing this issue due to age and because its references are long out of date. For example, distributed tests now have their own jobs.


Question: // with the mode. struct ModeUnsignedBoolPair match = {0, false}; <del> match = reduceBlockN<struct ModeUnsignedBoolPair, MatchReduceOp<struct ModeUnsignedBoolPair>, 2> </del> <ins> match = reduceBlockWithNThreadLocalReductions<struct ModeUnsignedBoolPair, MatchReduceOp<struct ModeUnsignedBoolPair>, 2> </ins> (ubpmem, ubpp, sliceSize, MatchReduceOp<struct ModeUnsignedBoolPair>(), match); // Finally, we have the mode, and an index where it occurs. We use a single thread

Not sure what is the reason for these errors, any suggestions? I suspect the clang version is not supported? Here is the
clang have more strict checking , pull request will fixed it.
fixed, thanks to


Question: } else { // transposed if (input.size(1) != weight.size(0)) { std::stringstream ss; <del> ss << "Given transposed=" << transposed << ", weight" << weight.sizes() << ", so expected input" << input.sizes() << " to have " </del> <ins> ss << "Given transposed=" << transposed << ", weight of size " << weight.sizes() << ", expected input" << input.sizes() << " to have " </ins> << weight.size(0) << " channels, but got " << input.size(1) << " channels instead"; throw std::runtime_error(ss.str());

got the following err msg with conv2D and ver 0.4: RuntimeError: expected stride to be a single integer value or a list of 1 values to match the convolution dimensions, but got stride=[1, 1]. I was not able to see the problem so I uninstalled 0.4 and installed 0.3.1 Still got an err but this time it said that the expected input should be a 4d tensor and it got a 3D tensor. This helped understand the issue and I fixed it (just add a dimension). reinstalled 0.4 and its working (no surprise...). I think 0.4 should have the same err feedback otherwise its really hard to undrstand the problem How you installed PyTorch (conda, pip, source): conda Build command you used (if compiling from source): OS: macos PyTorch version: 0.4 Python version: 3.5 CUDA/cuDNN version: no GPU models and configuration: GCC version (if compiling from source): CMake version: Versions of any other relevant libraries:
Can you please post a small self-contained code snippet that would let us reproduce the problem?
The strange error messge was also mentioned a few times in the forum. Here is a code snippet for PyTorch :
This is because we direct all conv ops to and infer dim + throw error message there.
Hmm that's not great. We might want to pass the expected dimensionality of the convolution to the generic implementation, so that we can improve the error messages.
Thx for the responses. Just a reminder, for the same err the provided err msg in version 0.3.1 was very informative
Yes, this should definitely be fixed. Sorry about this!
The error message of seems to be a bit misleading, too. Code: Should I create a new issue or is it related to the current one?
that appears fixed on master! :D


Question: if (weight_dim != k) { std::stringstream ss; <del> ss << "Expected " << k << "-dimensional weight for " << k << "-dimensional input " << input.sizes() << ", but got weight of size " << weight.sizes() << " instead"; </del> <ins> ss << "Expected " << weight_dim << "-dimensional input for " << weight_dim << "-dimensional weight " << weight.sizes() << ", but got input of size " << input.sizes() << " instead"; </ins> throw std::runtime_error(ss.str()); } if (weight.size(0) < groups) {


Question: return Subscript(base, [build_SliceExpr(ctx, base, expr.slice)]) elif sub_type is ast.ExtSlice: return Subscript(base, build_ExtSlice(ctx, base, expr.slice)) <ins> elif sys.version_info >= (3, 9):  # In Python3.9 array indicies are not wrapped in ast.Index if sub_type is ast.Tuple: # N-dimensional indexing using Tuple: x[(i, j, k)] is equivalent to x[i, j, k] indices = [] for index_expr in expr.slice.elts: if isinstance(index_expr, ast.Slice): indices.append(build_SliceExpr(ctx, base, index_expr)) else: indices.append(build_expr(ctx, index_expr)) return Subscript(base, indices) return Subscript(base, [build_expr(ctx, expr.slice)]) </ins> else:  # Ellipsis (can only happen in Python 2) raise NotSupportedError(base.range(), "ellipsis is not supported")

Following example: fails in Python-3.9 with cc
And the reason for that is very simple:
Another interesting offender:


Question: const float lr = *lr_ptr; if (!nesterov) { CUDA_1D_KERNEL_LOOP(i, N) { <del> moment_out[i] = mu * moment[i] * lr * grad[i]; </del> <ins> moment_out[i] = mu * moment[i] + lr * grad[i]; </ins> param_out[i] = param[i] - moment_out[i]; } } else {

+69 I read: To me, it should be: The CPU code is not affected.
The two lines of code are identical?
No, one is mu times moment time lr times grad instead of mu times moment plus lr times grad.
I see. Send a PR? :)
Will do :)


Question: if isinstance(expr.slice.value, ast.Tuple): # N-dimensional indexing using Tuple: x[(i, j, k)] is equivalent to x[i, j, k] # XXX: Indexing using a list is **different**! It triggers advanced indexing. <del> indices = [] for index_expr in expr.slice.value.elts: indices.append(build_expr(ctx, index_expr)) </del> <ins> indices = [build_expr(ctx, index_expr) for index_expr in expr.slice.value.elts] </ins> return Subscript(base, indices) else: return Subscript(base, [build_expr(ctx, expr.slice.value)])


Question: 'zero-dimensional.*cannot be concatenated'): torch.cat([x, y]) <del> def test_cat_empty(self): </del> <ins> @staticmethod def _test_cat_empty(self, use_cuda=False): </ins> # FIXME: this is legacy behavior and should be removed # when we support empty tensors with arbitrary sizes <del> x = torch.randn(4, 3, 32, 32) empty = torch.randn(0) </del> <ins> if use_cuda: dtype = torch.cuda.float32 else: dtype = torch.float32 x = torch.randn((4, 3, 32, 32), dtype=dtype) empty = torch.randn((0,), dtype=dtype) </ins> res1 = torch.cat([x, empty], dim=1) res2 = torch.cat([empty, x], dim=1) self.assertEqual(res1, res2) <del> conv = torch.nn.Conv2d(3, 3, kernel_size=1) </del> <ins> conv = torch.nn.Conv2d(3, 3, kernel_size=1).float() if use_cuda: conv = conv.cuda() </ins> res1 = torch.cat([conv(x), empty], dim=1) res2 = torch.cat([empty, conv(x)], dim=1) self.assertEqual(res1, res2)

gdb points that the error might be in trying to get a size from an empty tensor: I'm using PyTorch version 0.4.0a0+
There's a check that should exclude zero-dim tensors from ( should be 0 in this case), so I'm wondering why that's not happening right now... edit: Nevermind, I was running an old build. I pulled the latest master and with being an empty tensor (with shape (0,)), cat crashes.
Related: We should probably rewrite to better handle these cases
Okay, I found the bug. The CUDA version of check doesn't check the case where the input contains all empty tensors, while the CPU version does. I'll put up a fix soon.


Question: { tcntr = y*iwidth + x; real val = *(ip + tcntr); <del> if (val > maxval) </del> <ins> if ((val > maxval) || isnan(val)) </ins> { maxval = val; maxindex = tcntr;


Question: { tcntr = y*iwidth + x; real val = *(ip + tcntr); <del> if (val > maxval) </del> <ins> if ((val > maxval) || isnan(val)) </ins> { maxval = val; maxindex = tcntr;


Question: t2 = torch.from_numpy(t.numpy().transpose()) self.assertEqual(t1, t2) <ins> def test_inplace_division(self): t = torch.rand(5, 5) id_before = id(t) t /= 2 id_after = id(t) self.assertEqual(id_before, id_after) </ins> # Functions to test negative dimension wrapping METHOD = 1 INPLACE_METHOD = 2

This is an ipython session. Note that the doesn't remain the same for /= even though it works for div_
Just to be clear, the reason this is an issue is that it means that functions can't do inplace operations on function arguments.
Also, this works fine for +=


Question: Args: input (Tensor): the tensor to compare other (Tensor or float): the tensor or value to compare <del> out (Tensor, optional): the output tensor. Must be a `ByteTensor` or the same type as `input`. </del> <ins> out (Tensor, optional): the output tensor. Must be a `ByteTensor` </ins> Returns: Tensor: A ``torch.ByteTensor`` containing a 1 at each location where comparison is true

[pytorch] The docs say that for tensor comparison operators (gt,lt etc) it should be possible to pass out argument typed as input (), yet when I try to do it, I hit an error Should the docs be fixed, or is it a bug?
If it's really useful we can add it back; but we'll fix the docs for now


Question: blockIdx.x; } <del> // Block-wide reduction in shared memory helper; only threadIdx.x == 0 will // return the reduced value template <typename T, typename ReduceOp> __device__ T reduceBlock(T* smem, int numVals, T threadVal, ReduceOp reduceOp, T init) { </del> <ins> // Reduce N values concurrently, i.e. suppose N = 2, and there are 4 threads: // (1, 2), (3, 4), (5, 6), (7, 8), then the return in threadVals for thread 0 // is (1 + 3 + 5 + 7, 2 + 4 + 6 + 8) = (16, 20) template <typename T, typename ReduceOp, int N> __device__ void reduceNValuesInBlock(T *smem, T threadVals[N], int numVals, ReduceOp reduceOp, T init) { </ins> if (numVals == 0) { <del> return init; </del> <ins> #pragma unroll for (int i = 0; i < N; ++i) { threadVals[i] = init; } return; </ins> } <ins> // We store each of the N values contiguously, so if N = 2, all values for // the first threadVal for each thread in the block are stored followed by // all of the values for the second threadVal for each thread in the block </ins> if (threadIdx.x < numVals) { <del> smem[threadIdx.x] = threadVal; </del> <ins> #pragma unroll for (int i = 0; i < N; ++i) { smem[i * numVals + threadIdx.x] = threadVals[i]; } </ins> } <del> // First warp will perform reductions across warps </del> __syncthreads(); <del> if ((threadIdx.x / warpSize) == 0) { T r = threadIdx.x < numVals ? smem[threadIdx.x] : init; </del> <ins> // Number of lanes in the final reduction --> this is used to determine // where to put the outputs of each of the n things we are reducing. If // nLP = 32, then we have the 32 outputs for the first threadVal, // followed by the 32 outputs for the second threadVal, etc. int numLanesParticipating = min(numVals, warpSize); if (numVals > warpSize && ((threadIdx.x / warpSize) == 0 )) { #pragma unroll for (int i = 0; i < N; ++i) { threadVals[i] = threadIdx.x < numVals ? threadVals[i] : init; } </ins> for (int i = warpSize + threadIdx.x; i < numVals; i += warpSize) { <del> r = reduceOp(r, smem[i]); </del> <ins> #pragma unroll for (int j = 0; j < N; ++j) { threadVals[j] = reduceOp(threadVals[j], smem[j * numVals + i]); } </ins> } <del> smem[threadIdx.x] = r; </del> <ins> #pragma unroll for (int i = 0; i < N; ++i) { smem[i * numLanesParticipating + threadIdx.x] = threadVals[i]; } </ins> } <del> // First thread will perform reductions across the block </del> __syncthreads(); <del> T r = init; </del> if (threadIdx.x == 0) { <del> r = smem[0]; int numLanesParticipating = min(numVals, warpSize); </del> if (numLanesParticipating == 32) { <del> // Unroll for warpSize == 32 and numVals >= 32 </del> #pragma unroll <del> for (int i = 1; i < 32; ++i) { r = reduceOp(r, smem[i]); </del> <ins> for (int i = 0; i < N; ++i) { #pragma unroll for (int j = 1; j < 32; ++j) { threadVals[i] = reduceOp(threadVals[i], smem[i * 32 + j]); } </ins> } } else { <del> for (int i = 1; i < numLanesParticipating; ++i) { r = reduceOp(r, smem[i]); </del> <ins> #pragma unroll for (int i = 0; i < N; ++i) { for (int j = 1; j < numLanesParticipating; ++j) { threadVals[i] = reduceOp(threadVals[i], smem[i * numVals + j]); } </ins> } } } <ins> } </ins> <del> return r; </del> <ins> // Block-wide reduction in shared memory helper; only threadIdx.x == 0 will // return the reduced value template <typename T, typename ReduceOp> __device__ T reduceBlock(T* smem, int numVals, T threadVal, ReduceOp reduceOp, T init) { reduceNValuesInBlock<T, ReduceOp, 1>(smem, &threadVal, numVals, reduceOp, init); return threadVal; </ins> } // Block-wide reduction where each thread locally reduces N // values before letting a single warp take over - assumes // threadVals is in registers, not shared memory template <typename T, typename ReduceOp, int N> <del> __device__ T reduceBlockN(T *smem, </del> <ins> __device__ T reduceBlockWithNThreadLocalReductions(T *smem, </ins> T threadVals[N], int numVals, ReduceOp reduceOp,


Question: #include "THCTensor.hpp" #include "THCHalf.h" #include "THCHalfAutoNumerics.cuh" <ins> #include "THCNumerics.cuh" </ins> #include "common.h" // kernels borrowed from Caffe


Question: #include "THCTensor.hpp" #include "THCHalf.h" #include "THCHalfAutoNumerics.cuh" <ins> #include "THCNumerics.cuh" </ins> #include "common.h" // kernels borrowed from Caffe


Question: # This should work though l2.weight = Variable(torch.randn(10, 10)) <ins> def test_embedding_padding_idx(self): embedding = nn.Embedding(10, 20, padding_idx = 0) input = Variable(torch.LongTensor([[0,2,4,5],[4,3,0,9]])) output = embedding(input) self.assertEqual(output[0][0].sum().data[0], 0) self.assertEqual(output[1][2].sum().data[0], 0) </ins> def test_Dropout(self): input = torch.Tensor(1000) self._test_dropout(nn.Dropout, input)

The following code, which repeatedly exports a model with , has a memory leak. During the export, every tensor parameter in is cloned once and then immediately leaked forever, without ever being collected by the GC. It's not the underlying buffer that's cloned, it's the lightweight wrapper object itself. Still, for long running processes that often export networks in this manner this is a unbounded memory leak that eventually results in OOM errors. I've reproduced this issue on both Linux and Windows, with pytorch versions and respectively. The final five lines inside the for loop are to debug what happens, they are not neccesary to reproduce the issue. forces a gc collection cycle, ensuring we're not accidentally counting dead objects show the total amount of objects that exist for each type for all objects whose amount has increased. From this we can see that we're leaking 2 additional tensors per that the tensors we're leaking have shapes and , so they're just the weight and bias of the linear that the underlying buffer is always the same, so only the shallow class instance is being that nothing is pointing to these newly created objects, so they should be collected. Example output after running for a while: seems closely related but is more about a temporary doubling in memory, this issue is about a permanent memory leak. was closed as a duplicate of the previous issue, but better matches this issue. Collecting environment information. PyTorch version: 1.10.0 Is debug build: False CUDA used to build PyTorch: 11.3 ROCM used to build PyTorch: N/A OS: Ubuntu 18.04.5 LTS (x8664) GCC version: (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0 Clang version: 6.0.0-1ubuntu2 (tags/RELEASE600/final) CMake version: version 3.10.2 Libc version: glibc-2.17 Python version: 3.7.11 (default, Jul 27 2021, 14:32:16) [GCC 7.5.0] (64-bit runtime) Python platform: Linux-5.15.0-41-generic-x8664-with-debian-buster-sid Is CUDA available: True CUDA runtime version: 11.3.


Question: # This should work though l2.weight = Variable(torch.randn(10, 10)) <ins> def test_embedding_padding_idx(self): embedding = nn.Embedding(10, 20, padding_idx = 0) input = Variable(torch.LongTensor([[0,2,4,5],[4,3,0,9]])) output = embedding(input) self.assertEqual(output[0][0].sum().data[0], 0) self.assertEqual(output[1][2].sum().data[0], 0) </ins> def test_Dropout(self): input = torch.Tensor(1000) self._test_dropout(nn.Dropout, input)

109 GPU models and configuration: GPU 0: NVIDIA GeForce RTX 3080 Ti GPU 1: NVIDIA GeForce RTX 3080 Ti Nvidia driver version: 515.48.07 cuDNN version: Probably one of the following: /usr/lib/x8664-linux- /usr/lib/x8664-linux- /usr/lib/x8664-linux- /usr/lib/x8664-linux- /usr/lib/x8664-linux- /usr/lib/x8664-linux- /usr/lib/x8664-linux- HIP runtime version: N/A MIOpen runtime version: N/A Is XNNPACK available: True Versions of relevant libraries: [pip3] numpy==1.21.2 [pip3] torch==1.10.0 [pip3] torchelastic==0.2.0 [pip3] torchtext==0.11.0 [pip3] torchvision==0.11.0 [conda] blas 1.0 mkl [conda] cudatoolkit 11.3.1 ha36c4319 nvidia [conda] ffmpeg 4.3 hf484d3e0 pytorch [conda] mkl 2021.3.0 h06a4308520 [conda] mkl-service 2.4.0 py37h7f8727e0 [conda] mklfft 1.3.1 py37hd3c417c0 [conda] mklrandom 1.2.2 py37h51133e40 [conda] numpy 1.21.2 py37h20f2e390 [conda] numpy-base 1.21.2 py37h79a11010 [conda] pytorch 1.10.0 py3.7cuda11.3cudnn8.2.00 pytorch [conda] pytorch-mutex 1.0 cuda pytorch [conda] torchelastic 0.2.0 pypi0 pypi [conda] torchtext 0.11.0 py37 pytorch [conda] torchvision 0.11.0 py37cu113 pytorch
I'm also observing something like this without JIT, although I'm not sure it's the same issue. The program works fine until I add ONNX export every time a checkpoint is saved. Once I do that, the GPU memory usage grows until it OOMs.


Question: # This should work though l2.weight = Variable(torch.randn(10, 10)) <ins> def test_embedding_padding_idx(self): embedding = nn.Embedding(10, 20, padding_idx = 0) input = Variable(torch.LongTensor([[0,2,4,5],[4,3,0,9]])) output = embedding(input) self.assertEqual(output[0][0].sum().data[0], 0) self.assertEqual(output[1][2].sum().data[0], 0) </ins> def test_Dropout(self): input = torch.Tensor(1000) self._test_dropout(nn.Dropout, input)

This issue is about lightweight Tensor objects being leaked, not the underlying (potentially GPU-side) buffer. I think your issue is a different one.
I encountered the same error, is there a solution to this problem?
Please validate with the latest release and re-summit an issue if you see the same thing. As we are moving away from torchscript minor leaks are unlikely to be fixed, but contribution is welcomed.


Question: z = torch.cat([x, y]) self.assertEqual(z.size(), (21, SIZE, SIZE)) <ins> def test_cat_empty(self): TestTorch._test_cat_empty(self, use_cuda=True) </ins> def test_bernoulli(self): x = torch.tensor([0, 1], dtype=torch.cuda.float32) self.assertEqual(x.bernoulli().tolist(), [0, 1])


Question: } } <del> static Tensor dispatch_clamp(const Tensor & self, Scalar min, Scalar max) { AutoNoGIL no_gil; AutoGPU auto_gpu(self); return self.clamp(min, max); } static Tensor dispatch_clamp_min(const Tensor & self, Scalar min) { AutoNoGIL no_gil; AutoGPU auto_gpu(self); return self.clamp_min(min); } static Tensor dispatch_clamp_max(const Tensor & self, Scalar max) { AutoNoGIL no_gil; AutoGPU auto_gpu(self); return self.clamp_max(max); } </del> // The Python clamp() syntax has to be mapped to one of three C++ functions static PyObject * THPVariable_clamp(PyObject* module, PyObject* args, PyObject* kwargs) { HANDLE_TH_ERRORS static PythonArgParser parser({ <del> "clamp(Tensor input, Scalar min=None, Scalar max=None)", </del> <ins> "clamp(Tensor input, Scalar min=None, Scalar max=None, *, Tensor out=None)", </ins> }); <del> ParsedArgs<3> parsed_args; </del> <ins> ParsedArgs<4> parsed_args; </ins> auto r = parser.parse(args, kwargs, parsed_args); if (!r.isNone(1) && !r.isNone(2)) { <del> return THPVariable_Wrap(dispatch_clamp(r.tensor(0), r.scalar(1), r.scalar(2))); </del> <ins> if (!r.isNone(3)) { return wrap(dispatch_clamp(r.tensor(0), r.scalar(1), r.scalar(2), r.tensor(3))); } else { return wrap(dispatch_clamp(r.tensor(0), r.scalar(1), r.scalar(2))); } </ins> } else if (!r.isNone(1)) { <del> return THPVariable_Wrap(dispatch_clamp_min(r.tensor(0), r.scalar(1))); </del> <ins> if (!r.isNone(3)) { return wrap(dispatch_clamp_min(r.tensor(0), r.scalar(1), r.tensor(3))); } else { return wrap(dispatch_clamp_min(r.tensor(0), r.scalar(1))); } </ins> } else if (!r.isNone(2)) { <del> return THPVariable_Wrap(dispatch_clamp_max(r.tensor(0), r.scalar(2))); </del> <ins> if (!r.isNone(3)) { return wrap(dispatch_clamp_max(r.tensor(0), r.scalar(2), r.tensor(3))); } else { return wrap(dispatch_clamp_max(r.tensor(0), r.scalar(2))); } </ins> } else { throw std::runtime_error("At least one of 'min' or 'max' must not be None"); } <ins> Py_RETURN_NONE; </ins> END_HANDLE_TH_ERRORS }

In PyTorch master:
Added clamp's output support in pre-template code.


Question: Args: input (Tensor): the tensor to compare other (Tensor or float): the tensor or value to compare <del> out (Tensor, optional): the output tensor that must be a `ByteTensor` or the same type as `input` </del> <ins> out (Tensor, optional): the output tensor that must be a `ByteTensor` </ins> Returns: Tensor: A ``torch.ByteTensor`` containing a 1 at each location where comparison is true.


Question: def __idiv__(self, other): return self.div_(other) <ins> __itruediv__ = __idiv__ </ins> def __mod__(self, other): return self.remainder(other)


Question: // Lua indices begin at 1 IndexType dstIndex_ = indices.data[IndexToOffset<int64_t, IndexType, IdxDim>::get(dstIndex, indices)] - TH_INDEX_BASE; <del> assert(dstIndex < dstFillDimSize); </del> <ins> assert(dstIndex_ < dstFillDimSize); </ins> // We stride over the output ignoring the indexed dimension // (innerSize), whose offset calculation is handled differently

It seems that can change memory outside x, when x is a cuda tensor. If x is non-cuda tensor, we get: In contrast, when x is cuda tensor, does not make any error It's hard to share the whole code, but I have noticed that such operation outside a tensor did affect the performance of existing network, so I'm afraid that this op can change arbitrary memory on GPU which can be dangerous. Could you check this out?
This snippet is fine - it's enough for us to reproduce the problem. It appears we're missing some out-of-bounds checks (we have them for other indexing functions). Thanks for reporting.
working on this


Question: tW = targetTensor.size(tDims - 1) adjW = self._calculateAdj(tW, self.kW, self.padW, self.dW) adjH = self._calculateAdj(tH, self.kH, self.padH, self.dH) <del> if self.finput is None: </del> <ins> if not hasattr(self, 'finput') or self.finput is None: </ins> self.finput = input[0].new() <del> if self.fgradInput is None: </del> <ins> if not hasattr(self, 'fgradInput') or self.fgradInput is None: </ins> self.fgradInput = input[0].new() else: <del> if self.finput is None: </del> <ins> if not hasattr(self, 'finput') or self.finput is None: </ins> self.finput = input.new() <del> if self.fgradInput is None: </del> <ins> if not hasattr(self, 'fgradInput') or self.fgradInput is None: </ins> self.fgradInput = input.new() inputTensor = self._makeContiguous(inputTensor)

I am new to pythonwhen i solve the promblem with the help below I find some confusion in the code I set ‘dimension=1self.dimension = dimension’it seem ok for mebut i don’t kown how the value of ’dimension‘ was initialled. Thank you !
I already Konw it comes from 'module = JoinTable(dimension, nInputDims)' But when I convert the model to pytorch , error appears: Traceback (most recent call last): File "", line 173, in <moduleGnetf =generator.forward(input) File "/usr/local/lib/python2.7/dist-", line 33, in forward return self.updateOutput(input) File "/usr/local/lib/python2.7/dist-", line 36, in updateOutput currentOutput = module.updateOutput(currentOutput) File "/usr/local/lib/python2.7/dist-", line 37, in updateOutput (dim, offset, (dim)).copy_(currentOutput) RuntimeError: inconsistent tensor size at /home/lxl/pytorch-master/torch/lib/TH/generic/THTensorCopy.c:51
I Use "generator.modules[0] = nn.JoinTable(1)",it was fine ,but error again: Traceback (most recent call last): File "", line 171, in <moduleGnetf =generator.forward(input) File "/usr/local/lib/python2.7/dist-", line 33, in forward return self.updateOutput(input) File "/usr/local/lib/python2.7/dist-", line 36, in updateOutput currentOutput = module.updateOutput(currentOutput) File "/usr/local/lib/python2.7/dist-", line 96, in updateOutput if is None: AttributeError: 'SpatialFullConvolution' object has no attribute 'finput'
How old is the Lua model file you're trying to import? Can you please try to load it in Lua, save again, and load it in PyTorch? Also, please update PyTorch to the newest version.
The model is convert from the cudnn model trained by myseft the code below is the convert code BTWmy torch was installed on 17th Dec,2016 the pytorch version i use Metadata-Version: 1.0 Name: torch Version: 0.1.10+ I Build it from source today


Question: THPByteOrder::THP_LITTLE_ENDIAN, to_convert); } <del> SYSCHECK(write(fd, data, to_convert * sizeof(real))); </del> <ins> SYSCHECK(write(fd, le_buffer.get(), to_convert * sizeof(real))); </ins> } } }

The problem is here: You can't write an arbitrary number of bytes. See . On my system the limit seems to be 2GB, YMMV. To be safe, you probably want to fix the read call as well at , because there's an SSIZE_MAX limit.


Question: <del> Subproject commit 9f6a636e547fc70a02fa48436449aad67080698f </del> <ins> Subproject commit add56ccdcac23a6c522a2c1174a866e293c61dab </ins>

Pybind11 has a bugfix here: which is not included in pytorch master. In brief, the bug causes two python modules, when both compiled with buggy version of pybind11, to conflict and crash at import. I've last week when debugging its conflict with pytorch. Hope pytorch can also upgrade to avoid potential conflict with other libraries.


Question: self.assertEqual(output[0][0].sum().data[0], 0) self.assertEqual(output[1][2].sum().data[0], 0) <ins> def test_embedding_max_norm(self): embedding = nn.Embedding(22, 5, max_norm=1.0) input = Variable(torch.LongTensor([2, 8, 8, 6])) output = embedding(input) self.assertEqual(output[1], output[2]) self.assertTrue(output.data.norm(p=2, dim=1).le(1).all()) @unittest.skipIf(not TEST_CUDA, "CUDA unavailable") def test_embedding_max_norm_cuda(self): embedding = nn.Embedding(22, 5, max_norm=1.0).cuda() input = Variable(torch.LongTensor([2, 8, 8, 6])).cuda() output = embedding(input) self.assertEqual(output[1], output[2]) self.assertTrue(output.data.norm(p=2, dim=1).le(1).all()) </ins> def test_embedding_functional(self): a = Variable(torch.LongTensor([ [1, 3, 2],

The output is also incorrect. It's the output from the sorted indices, instead of the user specified indices. Reported by


Question: for(ih = 0; ih < kH; ih++) { for(iw = 0; iw < kW; iw++) { T val = ptr_input[iw*istrideW]; <del> if (val > max) { </del> <ins> if ((val > max) || THCNumerics<T>::isnan(val)) { </ins> max = val; argmax = (ih+istartH)*isizeW + iw+istartW; }

max pooling functions are not consistent with max functions. Below an example, every max pooling (be it 1d, 2d or 3d, adaptive or not) acts the same, on cpu or on cuda. Essentially, there are two fondamental differences : max pooling of all values is while for it's max pooling of nan and valid values is valid values, which means s get ignored, while for , as soon as there is a value, the result is . More generally, choosing explicetely how to deal with as in numpy () could be a solution, but maybe this is related to CuDNN's max pooling ? Built from latest sources (as of 05/17) PyTorch version: 0.5.0a0+ Is debug build: No CUDA used to build PyTorch: 9.1.85 OS: Ubuntu 16.04.4 LTS GCC version: (Ubuntu 5.4.0-6ubuntu1~16.04.9) 5.4.0 CMake version: version 3.9.4 Python version: 3.6 Is CUDA available: Yes CUDA runtime version: 9.1.85 GPU models and configuration: GPU 0: Quadro M1000M Nvidia driver version: 390.30 cuDNN version: Probably one of the following: /usr/lib/x8664-linux- /usr/lib/x8664-linux-gnu/libcudnnstaticv7.a Versions of relevant libraries: [conda] magma-cuda91 2.3.0 1 pytorch [conda] torch 0.5.0a0+ <pip[conda] torch 0.3.1b0+ <pip[conda] torch 0.5.0a0+ <pip
Hi, . Thank you for providing the example. I'll take a look.
Hmhm. I seem to get a double free in the ...
Another question: Do we really want to provide an option to ignore NaN? My view is that "if you have nan in your net, you're screwed", so I would just return NaN there, personally.
My practical usecase is for KITTI groundtruth depthmaps and FlowMaps which are 2D sparse arrays.


Question: for(ih = 0; ih < kH; ih++) { for(iw = 0; iw < kW; iw++) { T val = ptr_input[iw*istrideW]; <del> if (val > max) { </del> <ins> if ((val > max) || THCNumerics<T>::isnan(val)) { </ins> max = val; argmax = (ih+istartH)*isizeW + iw+istartW; }

For an algorithm that use FlowNet-like architecture that outputs predictions at multiple scale levels, we can either compare predictions to downscaled GT or upscaled predictions to GT. The first being obviously less computationally expensive, the ignore NaN would help downscaling such sparse 2D maps. For the moment we do something I find ugly, you can see it Essentially it zeroes the s, takes the map and construct two maps of positive and negative values which are then maxpooled and back together. I am actually open for a strict "no-nan" policy on pooling functions, but in that case better enforce it before someone writes a code that tries to benefit from maxpooling ignore s feature/bug and if you have a clever way of pooling sparse 2D tensors, I'm open to it, but I guess it's a topic for pytorch forums ;)
Personally, I think it is more sane have NaN -NaN in the pooling and offer a parametrizable (where you get to pick the values) that does or so.
So the proposed fix does NaN -NaN similar to max. I didn't try to fix gradients for the nan case. This would involve keeping the values and I don't think that is worth it (in particular because I would not expect the pooling layer to be last, and otherwise we'd probably get NaN as grad_out). If you are reasonably happy with it, I'd move it to a PR.
+1 for NaN -NaN, "abyssus abyssum invocat" We could also add an optional mask which would be a ByteTensor of the same size, specifying whether or not the considered pixel is used for the pooling, that could be used for any kind of pulling. An functionality would then be to provide the mask
I could work with that. to continue to be picky, the potential drawback is that it won't work for other pooling methods such as average pooling or median pooling. The problem here is that the max operation inherently ignores nonmax values which can be leveraged for "ignore some pixels" operation, but it has a "non universal" feel to it since it woun't be as easy for other kinds of 2D operations.
To be clear, the fix is good for me, but I figured a related discussion on selective pooling (whether to ditch NaN values or anything you want to ignore) could happen (maybe not on this issue ?)


Question: <ins> #define __STDC_FORMAT_MACROS </ins> #include <Python.h> #ifdef _MSC_VER #include <Windows.h>

Hi There, I'm trying to install Pytorch as a module on my university's computing cluster. The node uses CentOS (6.4) x8664. I install running these commands: module load gcclib/5.2.0 cmake/3.8.2 module load gcc/5.2.0 module load anaconda3/4.0.0 export CMAKEPREFIXPATH="(which conda))/../" export NOCUDA=1 git clone --recursive cd pytorch/ python install --prefix=${HOME} Things look fine as it installs but then I run into the following errors, ending in the process terminating with error relating to GCC? I've tried with version GCC 6.2.0 and the same result occurs. Not sure what to even try to fix this! Thanks for any help you can provide!
Same here, with gcc 5.4.0
I don't think building from source works very well with gcc 5.4. Could you install gcc 4.9 and try compiling?
I get this error instead when using gcc 4.9.0, doesn't make it very far at all.
Hi I think this is broken after / due to
I have the same problem ( etc.) with gcc 4.8.5 on Linux with the current head ( ) What works for me is to add at the beginning of the following four files: (see also -- this can be turned into a pull request very easily) This fix is the same as but for different files.
This worked for me Thank you.
fixed in latest master, thanks to


Question: // fast track for bytes and little endian if (sizeof(real) == 1 || THP_nativeByteOrder() == THPByteOrder::THP_LITTLE_ENDIAN) { <del> SYSCHECK(read(fd, data, sizeof(real) * storage->size)); </del> <ins> char *bytes = (char *) data; uint64_t remaining = sizeof(real) * storage->size; while (remaining > 0) { ssize_t result = read(fd, bytes, remaining); if (result < 0) throw std::system_error(result, std::system_category()); bytes += result; remaining -= result; } </ins> } else { long buffer_size = std::min(size, (long)5000); std::unique_ptr<uint8_t[]> le_buffer(new uint8_t[buffer_size * sizeof(real)]); <del> for (long i = 0; i < size; i += buffer_size) { </del> <ins> for (int64_t i = 0; i < size; i += buffer_size) { </ins> size_t to_convert = std::min(size - i, buffer_size); SYSCHECK(read(fd, le_buffer.get(), sizeof(real) * to_convert)); if (sizeof(real) == 2) {


Question: Args: input (Tensor): the tensor to compare other (Tensor or float): the tensor or value to compare <del> out (Tensor, optional): the output tensor that must be a `ByteTensor` or the same type as :attr:`input` </del> <ins> out (Tensor, optional): the output tensor that must be a `ByteTensor` </ins> Returns: Tensor: A ``torch.ByteTensor`` containing a 1 at each location where comparison is true


Question: "aten::set_grad_enabled(bool val) -> ()", [](Stack* stack) { torch::GradMode::set_enabled(pop(stack).toBool()); <del> push(stack, IValue()); </del> }, aliasAnalysisConservative()), });

Recently, we are testing PyTorch 1.7 as we need the with-statement support, so that our algorithm which contains a custom module using can be deployed to the production environment. During the testing, we encountered the following assertion failure: Steps to reproduce the behavior: the following Python code to create a serialized PyTorch module. a C++ test program. the test program with the following command: in the current directory. should terminate with exit code 0 instead of throw an exception. Collecting environment information... PyTorch version: 1.7.0.dev20200922+cpu Is debug build: True CUDA used to build PyTorch: None ROCM used to build PyTorch: N/A OS: CentOS Linux 7 (Core) (x8664) GCC version: (GCC) 4.8.5 (Red Hat 4.8.5-39) Clang version: Could not collect CMake version: Could not collect Python version: 3.7 (64-bit runtime) Is CUDA available: False CUDA runtime version: No CUDA GPU models and configuration: No CUDA Nvidia driver version: No CUDA cuDNN version: No CUDA HIP runtime version: N/A MIOpen runtime version: N/A Versions of relevant libraries: [pip3] numpy==1.19.2 [pip3] torch==1.7.0.dev20200922+cpu [pip3] torchvision==0.7.0+cpu [conda] Could not collect cc


Question: } } <ins> // If all inputs are empty tensors, return an empty tensor if (notEmptyTensor == NULL) { return; } </ins> // In the event that the user specified -1 as the concat dimension, then // we want to pick the nDims as dimension to cat along (and thus nDims - 1 as the // value due to 0-based indexing). If the nDims is // 0 (i.e. we are catting all


Question: bottom_data += (n * channels + c) * height * width; for (int h = hstart; h < hend; h += dilation_h) { for (int w = wstart; w < wend; w += dilation_w) { <del> if (ScalarConvert<Dtype, AccType>::to(bottom_data[h * width + w]) > maxval) { </del> <ins> Dtype val = bottom_data[h * width + w]; if ((ScalarConvert<Dtype, AccType>::to(val) > maxval) || THCNumerics<Dtype>::isnan(val)) { </ins> maxidx = h * width + w; <del> maxval = ScalarConvert<Dtype, AccType>::to(bottom_data[maxidx]); </del> <ins> maxval = ScalarConvert<Dtype, AccType>::to(val); </ins> } } }


Question: bottom_data += (n * channels + c) * height * width; for (int h = hstart; h < hend; h += dilation_h) { for (int w = wstart; w < wend; w += dilation_w) { <del> if (ScalarConvert<Dtype, AccType>::to(bottom_data[h * width + w]) > maxval) { </del> <ins> Dtype val = bottom_data[h * width + w]; if ((ScalarConvert<Dtype, AccType>::to(val) > maxval) || THCNumerics<Dtype>::isnan(val)) { </ins> maxidx = h * width + w; <del> maxval = ScalarConvert<Dtype, AccType>::to(bottom_data[maxidx]); </del> <ins> maxval = ScalarConvert<Dtype, AccType>::to(val); </ins> } } }


Question: <ins> #define __STDC_FORMAT_MACROS </ins> #include <Python.h> #include <structmember.h>


Question: auto input = input_r.contiguous(); auto weight = weight_r; auto bias = bias_r; <del> auto k = input.ndimension(); </del> <ins> auto k = weight.ndimension(); </ins> int64_t dim = k - 2; if (dim <= 0) { <del> throw std::runtime_error("input has less dimensions than expected"); </del> <ins> throw std::runtime_error("weight should have at least two dimensions"); </ins> } ConvParams params;

got the following err msg with conv2D and ver 0.4: RuntimeError: expected stride to be a single integer value or a list of 1 values to match the convolution dimensions, but got stride=[1, 1]. I was not able to see the problem so I uninstalled 0.4 and installed 0.3.1 Still got an err but this time it said that the expected input should be a 4d tensor and it got a 3D tensor. This helped understand the issue and I fixed it (just add a dimension). reinstalled 0.4 and its working (no surprise...). I think 0.4 should have the same err feedback otherwise its really hard to undrstand the problem How you installed PyTorch (conda, pip, source): conda Build command you used (if compiling from source): OS: macos PyTorch version: 0.4 Python version: 3.5 CUDA/cuDNN version: no GPU models and configuration: GCC version (if compiling from source): CMake version: Versions of any other relevant libraries:
Can you please post a small self-contained code snippet that would let us reproduce the problem?
The strange error messge was also mentioned a few times in the forum. Here is a code snippet for PyTorch :
This is because we direct all conv ops to and infer dim + throw error message there.
Hmm that's not great. We might want to pass the expected dimensionality of the convolution to the generic implementation, so that we can improve the error messages.
Thx for the responses. Just a reminder, for the same err the provided err msg in version 0.3.1 was very informative
Yes, this should definitely be fixed. Sorry about this!
The error message of seems to be a bit misleading, too. Code: Should I create a new issue or is it related to the current one?
that appears fixed on master! :D


Question: for(iw = 0; iw < kW; iw++) { real val = *(ip + ih*istrideH + iw*istrideW); <del> if (val > maxval) </del> <ins> if ((val > maxval) || isnan(val)) </ins> { maxval = val; maxindex = (ih+istartH)*isizeW + (iw+istartW);


Question: for(iw = 0; iw < kW; iw++) { real val = *(ip + ih*istrideH + iw*istrideW); <del> if (val > maxval) </del> <ins> if ((val > maxval) || isnan(val)) </ins> { maxval = val; maxindex = (ih+istartH)*isizeW + (iw+istartW);


Question: gI = apply_fn<Transpose>(0, 1)(gIt); } } <ins> if (should_compute_output(0) && !ggO.defined()) ggO = at::zeros_like(gO); if (should_compute_output(1) && !gI.defined()) gI = at::zeros_like(input); if (should_compute_output(2) && !gW.defined()) gW = at::zeros_like(weight); </ins> return {ggO, gI, gW}; }

This is a test. Please ignore it. Edited.
<!-- validation-comment-start --<bodyHello there! From the DISABLED prefix in this issue title, it looks like you are attempting to disable a job in PyTorch CI. The information I have parsed is below: Job name: Credential: Within ~15 minutes, and all of its dependants will be disabled in PyTorch CI. Please verify that the job name looks correct. With great power comes great responsibility. </body<!-- validation-comment-end --


Question: 'expected a non-empty list of Tensors'): torch.cat([], dim=1) <ins> def test_cat_empty(self): self._test_cat_empty(self) </ins> def test_stack(self): x = torch.rand(2, 3, 4) y = torch.rand(2, 3, 4)

gdb points that the error might be in trying to get a size from an empty tensor: I'm using PyTorch version 0.4.0a0+
There's a check that should exclude zero-dim tensors from ( should be 0 in this case), so I'm wondering why that's not happening right now... edit: Nevermind, I was running an old build. I pulled the latest master and with being an empty tensor (with shape (0,)), cat crashes.
Related: We should probably rewrite to better handle these cases
Okay, I found the bug. The CUDA version of check doesn't check the case where the input contains all empty tensors, while the CPU version does. I'll put up a fix soon.


Question: } } <ins> // manual dispatch code for clamp inline Tensor dispatch_clamp(const Tensor & self, Scalar min, Scalar max) { AutoNoGIL no_gil; AutoGPU auto_gpu(self); return self.clamp(min, max); } inline Tensor dispatch_clamp_min(const Tensor & self, Scalar min) { AutoNoGIL no_gil; AutoGPU auto_gpu(self); return self.clamp_min(min); } inline Tensor dispatch_clamp_max(const Tensor & self, Scalar max) { AutoNoGIL no_gil; AutoGPU auto_gpu(self); return self.clamp_max(max); } inline Tensor & dispatch_clamp(const Tensor & self, Scalar min, Scalar max, Tensor result) { AutoNoGIL no_gil; AutoGPU auto_gpu(result); return at::clamp_out(result, self, min, max); } inline Tensor & dispatch_clamp_min(const Tensor & self, Scalar min, Tensor result) { AutoNoGIL no_gil; AutoGPU auto_gpu(result); return at::clamp_min_out(result, self, min); } inline Tensor & dispatch_clamp_max(const Tensor & self, Scalar max, Tensor result) { AutoNoGIL no_gil; AutoGPU auto_gpu(result); return at::clamp_max_out(result, self, max); } </ins> ${py_method_dispatch} }} // namespace torch::autograd


Question: for(ih = 0; ih < kH; ++ih) { for(iw = 0; iw < kW; ++iw) { T val = ptr_input[ih*istrideH + iw*istrideW]; <del> if (val > max) { </del> <ins> if ((val > max) || THCNumerics<T>::isnan(val)) { </ins> max = val; argmax = (it+istartT)*isizeH*isizeW + (ih+istartH)*isizeW + iw+istartW; }


Question: for(ih = 0; ih < kH; ++ih) { for(iw = 0; iw < kW; ++iw) { T val = ptr_input[ih*istrideH + iw*istrideW]; <del> if (val > max) { </del> <ins> if ((val > max) || THCNumerics<T>::isnan(val)) { </ins> max = val; argmax = (it+istartT)*isizeH*isizeW + (ih+istartH)*isizeW + iw+istartW; }


Question: index = t * inputH * inputW + h * inputW + w; Dtype val = inputData[index]; <del> if (max < val) </del> <ins> if ((max < val) || THCNumerics<Dtype>::isnan(val)) </ins> { max = val; maxIndex = index;


Question: index = t * inputH * inputW + h * inputW + w; Dtype val = inputData[index]; <del> if (max < val) </del> <ins> if ((max < val) || THCNumerics<Dtype>::isnan(val)) </ins> { max = val; maxIndex = index;


Question: res2[i] = max(min_val, min(max_val, res2[i])) self.assertEqual(res1, res2) <ins> out = m1.clone() torch.clamp(m1, min=min_val, max=max_val, out=out) self.assertEqual(out, res1) </ins> res1 = torch.clamp(m1, min=min_val) res2 = m1.clone() for i in iter_indices(res2): res2[i] = max(min_val, res2[i]) self.assertEqual(res1, res2) <ins> torch.clamp(m1, min=min_val, out=out) self.assertEqual(out, res1) </ins> res1 = torch.clamp(m1, max=max_val) res2 = m1.clone() for i in iter_indices(res2): res2[i] = min(max_val, res2[i]) self.assertEqual(res1, res2) <ins> torch.clamp(m1, max=max_val, out=out) self.assertEqual(out, res1) </ins> def test_pow(self): # [res] torch.pow([res,] x)


Question: if (!transposed) { if (input.size(1) != (weight.size(1) * groups)) { std::stringstream ss; <del> ss << "Given groups=" << groups << ", weight" << weight.sizes() << ", so expected input" << input.sizes() << " to have " </del> <ins> ss << "Given groups=" << groups << ", weight of size " << weight.sizes() << ", expected input" << input.sizes() << " to have " </ins> << (weight.size(1) * groups) << " channels, but got " << input.size(1) << " channels instead"; throw std::runtime_error(ss.str());


Question: struct ModeUnsignedPair max = {0, 0}; <del> max = reduceBlockN<struct ModeUnsignedPair, MaxReduceOp<struct ModeUnsignedPair>, 2> </del> <ins> max = reduceBlockWithNThreadLocalReductions<struct ModeUnsignedPair, MaxReduceOp<struct ModeUnsignedPair>, 2> </ins> (uupmem, uup, sliceSize, MaxReduceOp<struct ModeUnsignedPair>(), max); // Store the mode in shared memory for use in finding the mode in the input slice

Not sure what is the reason for these errors, any suggestions? I suspect the clang version is not supported? Here is the
clang have more strict checking , pull request will fixed it.
fixed, thanks to


Question: SYSCHECK(write(fd, &self->size, sizeof(long))); // fast track for bytes and little endian if (sizeof(real) == 1 || THP_nativeByteOrder() == THPByteOrder::THP_LITTLE_ENDIAN) { <del> SYSCHECK(write(fd, data, sizeof(real) * self->size)); </del> <ins> char *bytes = (char *) data; uint64_t remaining = sizeof(real) * self->size; while (remaining > 0) { ssize_t result = write(fd, bytes, remaining); if (result < 0) throw std::system_error(result, std::system_category()); bytes += result; remaining -= result; } </ins> } else { long buffer_size = std::min(self->size, (long)5000); std::unique_ptr<uint8_t[]> le_buffer(new uint8_t[buffer_size * sizeof(real)]); <del> for (long i = 0; i < self->size; i += buffer_size) { </del> <ins> for (int64_t i = 0; i < self->size; i += buffer_size) { </ins> size_t to_convert = std::min(self->size - i, buffer_size); if (sizeof(real) == 2) { THP_encodeInt16Buffer((uint8_t*)le_buffer.get(),


Question: def reset_parameters(self): self.weight.data.normal_(0, 1) <ins> if self.padding_idx is not None: self.weight.data[self.padding_idx].fill_(0) </ins> def forward(self, input): <del> return self._backend.Embedding(self.padding_idx, self.max_norm, self.norm_type, self.scale_grad_by_freq)(input, self.weight) </del> <ins> padding_idx = self.padding_idx if padding_idx is None: padding_idx = -1 return self._backend.Embedding(padding_idx, self.max_norm, self.norm_type, self.scale_grad_by_freq)(input, self.weight) </ins> # TODO: SparseLinear

The following code, which repeatedly exports a model with , has a memory leak. During the export, every tensor parameter in is cloned once and then immediately leaked forever, without ever being collected by the GC. It's not the underlying buffer that's cloned, it's the lightweight wrapper object itself. Still, for long running processes that often export networks in this manner this is a unbounded memory leak that eventually results in OOM errors. I've reproduced this issue on both Linux and Windows, with pytorch versions and respectively. The final five lines inside the for loop are to debug what happens, they are not neccesary to reproduce the issue. forces a gc collection cycle, ensuring we're not accidentally counting dead objects show the total amount of objects that exist for each type for all objects whose amount has increased. From this we can see that we're leaking 2 additional tensors per that the tensors we're leaking have shapes and , so they're just the weight and bias of the linear that the underlying buffer is always the same, so only the shallow class instance is being that nothing is pointing to these newly created objects, so they should be collected. Example output after running for a while: seems closely related but is more about a temporary doubling in memory, this issue is about a permanent memory leak. was closed as a duplicate of the previous issue, but better matches this issue. Collecting environment information. PyTorch version: 1.10.0 Is debug build: False CUDA used to build PyTorch: 11.3 ROCM used to build PyTorch: N/A OS: Ubuntu 18.04.5 LTS (x8664) GCC version: (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0 Clang version: 6.0.0-1ubuntu2 (tags/RELEASE600/final) CMake version: version 3.10.2 Libc version: glibc-2.17 Python version: 3.7.11 (default, Jul 27 2021, 14:32:16) [GCC 7.5.0] (64-bit runtime) Python platform: Linux-5.15.0-41-generic-x8664-with-debian-buster-sid Is CUDA available: True CUDA runtime version: 11.3.


Question: def reset_parameters(self): self.weight.data.normal_(0, 1) <ins> if self.padding_idx is not None: self.weight.data[self.padding_idx].fill_(0) </ins> def forward(self, input): <del> return self._backend.Embedding(self.padding_idx, self.max_norm, self.norm_type, self.scale_grad_by_freq)(input, self.weight) </del> <ins> padding_idx = self.padding_idx if padding_idx is None: padding_idx = -1 return self._backend.Embedding(padding_idx, self.max_norm, self.norm_type, self.scale_grad_by_freq)(input, self.weight) </ins> # TODO: SparseLinear

109 GPU models and configuration: GPU 0: NVIDIA GeForce RTX 3080 Ti GPU 1: NVIDIA GeForce RTX 3080 Ti Nvidia driver version: 515.48.07 cuDNN version: Probably one of the following: /usr/lib/x8664-linux- /usr/lib/x8664-linux- /usr/lib/x8664-linux- /usr/lib/x8664-linux- /usr/lib/x8664-linux- /usr/lib/x8664-linux- /usr/lib/x8664-linux- HIP runtime version: N/A MIOpen runtime version: N/A Is XNNPACK available: True Versions of relevant libraries: [pip3] numpy==1.21.2 [pip3] torch==1.10.0 [pip3] torchelastic==0.2.0 [pip3] torchtext==0.11.0 [pip3] torchvision==0.11.0 [conda] blas 1.0 mkl [conda] cudatoolkit 11.3.1 ha36c4319 nvidia [conda] ffmpeg 4.3 hf484d3e0 pytorch [conda] mkl 2021.3.0 h06a4308520 [conda] mkl-service 2.4.0 py37h7f8727e0 [conda] mklfft 1.3.1 py37hd3c417c0 [conda] mklrandom 1.2.2 py37h51133e40 [conda] numpy 1.21.2 py37h20f2e390 [conda] numpy-base 1.21.2 py37h79a11010 [conda] pytorch 1.10.0 py3.7cuda11.3cudnn8.2.00 pytorch [conda] pytorch-mutex 1.0 cuda pytorch [conda] torchelastic 0.2.0 pypi0 pypi [conda] torchtext 0.11.0 py37 pytorch [conda] torchvision 0.11.0 py37cu113 pytorch
I'm also observing something like this without JIT, although I'm not sure it's the same issue. The program works fine until I add ONNX export every time a checkpoint is saved. Once I do that, the GPU memory usage grows until it OOMs.


Question: def reset_parameters(self): self.weight.data.normal_(0, 1) <ins> if self.padding_idx is not None: self.weight.data[self.padding_idx].fill_(0) </ins> def forward(self, input): <del> return self._backend.Embedding(self.padding_idx, self.max_norm, self.norm_type, self.scale_grad_by_freq)(input, self.weight) </del> <ins> padding_idx = self.padding_idx if padding_idx is None: padding_idx = -1 return self._backend.Embedding(padding_idx, self.max_norm, self.norm_type, self.scale_grad_by_freq)(input, self.weight) </ins> # TODO: SparseLinear

This issue is about lightweight Tensor objects being leaked, not the underlying (potentially GPU-side) buffer. I think your issue is a different one.
I encountered the same error, is there a solution to this problem?
Please validate with the latest release and re-summit an issue if you see the same thing. As we are moving away from torchscript minor leaks are unlikely to be fixed, but contribution is welcomed.


Question: static inline __host__ __device__  char mul(char a, char b) { return a * b; } static inline __host__ __device__  char sub(char a, char b) { return a - b; } static inline __host__ __device__  char div(char a, char b) { return a / b; } <del> static inline __host__ __device__  char abs(char a) { return abs(a); } </del> <ins> static inline __host__ __device__  char abs(char a) { return ::abs((int)a); } </ins> }; template <>


Question: @staticmethod def _renorm(ctx, indices, weight, max_norm, norm_type): <del> if indices.dim() == 2: indices = indices.clone().view(-1) </del> <ins> # clone indices since LookupTable_renorm modifies it in-place </ins> ctx._backend.LookupTable_renorm( ctx._backend.library_state, <del> indices, </del> <ins> indices.clone().view(-1), </ins> weight, max_norm, norm_type


Question: static inline __host__ __device__  short mul(short a, short b) { return a * b; } static inline __host__ __device__  short sub(short a, short b) { return a - b; } static inline __host__ __device__  short div(short a, short b) { return a / b; } <del> static inline __host__ __device__  short abs(short a) { return abs(a); } </del> <ins> static inline __host__ __device__  short abs(short a) { return ::abs((int)a); } </ins> }; template <>


Question: >>> # an Embedding module containing 10 tensors of size 3 >>> embedding = nn.Embedding(10, 3) >>> # a batch of 2 samples of 4 indices each <del> >>> input = torch.Tensor([[1,2,4,5],[4,3,2,10]]) </del> <ins> >>> input = torch.LongTensor([[1,2,4,5],[4,3,2,9]]) >>> print(embedding(input)) >>> # example with padding_idx >>> embedding = nn.Embedding(10, 3, padding_idx=0) >>> input = torch.LongTensor([[0,2,0,5]]) </ins> >>> print(embedding(input)) """ <del> def __init__(self, num_embeddings, embedding_dim, padding_idx=-1, </del> <ins> def __init__(self, num_embeddings, embedding_dim, padding_idx=None, </ins> max_norm=None, norm_type=2, scale_grad_by_freq=False): self.num_embeddings = num_embeddings self.embedding_dim = embedding_dim


Question: >>> # an Embedding module containing 10 tensors of size 3 >>> embedding = nn.Embedding(10, 3) >>> # a batch of 2 samples of 4 indices each <del> >>> input = torch.Tensor([[1,2,4,5],[4,3,2,10]]) </del> <ins> >>> input = torch.LongTensor([[1,2,4,5],[4,3,2,9]]) >>> print(embedding(input)) >>> # example with padding_idx >>> embedding = nn.Embedding(10, 3, padding_idx=0) >>> input = torch.LongTensor([[0,2,0,5]]) </ins> >>> print(embedding(input)) """ <del> def __init__(self, num_embeddings, embedding_dim, padding_idx=-1, </del> <ins> def __init__(self, num_embeddings, embedding_dim, padding_idx=None, </ins> max_norm=None, norm_type=2, scale_grad_by_freq=False): self.num_embeddings = num_embeddings self.embedding_dim = embedding_dim


Question: >>> # an Embedding module containing 10 tensors of size 3 >>> embedding = nn.Embedding(10, 3) >>> # a batch of 2 samples of 4 indices each <del> >>> input = torch.Tensor([[1,2,4,5],[4,3,2,10]]) </del> <ins> >>> input = torch.LongTensor([[1,2,4,5],[4,3,2,9]]) >>> print(embedding(input)) >>> # example with padding_idx >>> embedding = nn.Embedding(10, 3, padding_idx=0) >>> input = torch.LongTensor([[0,2,0,5]]) </ins> >>> print(embedding(input)) """ <del> def __init__(self, num_embeddings, embedding_dim, padding_idx=-1, </del> <ins> def __init__(self, num_embeddings, embedding_dim, padding_idx=None, </ins> max_norm=None, norm_type=2, scale_grad_by_freq=False): self.num_embeddings = num_embeddings self.embedding_dim = embedding_dim


Question: with: submodules: false fetch-depth: 1 <del> - name: Setup Python 3.5 </del> <ins> - name: Setup Python 3.6 </ins> if: matrix.test_type == 'older_python_version' uses: actions/setup-python@v4 with: <del> python-version: '3.5' </del> <ins> python-version: '3.6' </ins> architecture: x64 check-latest: false cache: pip

Several this morning failed with (see for example): Not sure what is causing the outage, but it makes me wonder if perhaps it's time to retire Python-3.5 testing CI cc
Looks like pypi rolled out a new cert today:


Question: Args: num_embeddings: size of the dictionary of embeddings embedding_dim: the size of each embedding vector <del> padding_idx: If given, pads the output with zeros whenever it encounters the index. Default: -1 </del> <ins> padding_idx: If given, pads the output with zeros whenever it encounters the index. Default: None </ins> max_norm: If given, will renormalize the embeddings to always have a norm lesser than this Default: None norm_type: The p of the p-norm to compute for the max_norm option scale_grad_by_freq: if given, this will scale gradients by the frequency of the words in the dictionary.


Question: Args: num_embeddings: size of the dictionary of embeddings embedding_dim: the size of each embedding vector <del> padding_idx: If given, pads the output with zeros whenever it encounters the index. Default: -1 </del> <ins> padding_idx: If given, pads the output with zeros whenever it encounters the index. Default: None </ins> max_norm: If given, will renormalize the embeddings to always have a norm lesser than this Default: None norm_type: The p of the p-norm to compute for the max_norm option scale_grad_by_freq: if given, this will scale gradients by the frequency of the words in the dictionary.


Question: Args: num_embeddings: size of the dictionary of embeddings embedding_dim: the size of each embedding vector <del> padding_idx: If given, pads the output with zeros whenever it encounters the index. Default: -1 </del> <ins> padding_idx: If given, pads the output with zeros whenever it encounters the index. Default: None </ins> max_norm: If given, will renormalize the embeddings to always have a norm lesser than this Default: None norm_type: The p of the p-norm to compute for the max_norm option scale_grad_by_freq: if given, this will scale gradients by the frequency of the words in the dictionary.


Question: meant to be installed as pip packages) (default: False). relative_to (str, optional): path of the build file. Required when ``package is True``. It's best to use ``__file__`` for this argument. <del> kwargs: additional arguments that are passed to ffi to declar the </del> <ins> kwargs: additional arguments that are passed to ffi to declare the </ins> extension. See `Extension API reference`_ for details. .. _`Extension API reference`: https://docs.python.org/3/distutils/apiref.html#distutils.core.Extension

When trying to install Pytorch on my Mac by following the instructions I get What I did: ` I also tried Both approaches gave the same error. System: xcode-select version 2395. Version: macOS Monterey 12.3.1 (21E258) MacBook Pro (16-inch, 2019) Processor: 2,6 GHz 6-Core Intel Core i7 memory: 16 GB 2667 MHz DDR4 PyTorch version: N/A Is debug build: N/A CUDA used to build PyTorch: N/A ROCM used to build PyTorch: N/A OS: macOS 12.3.1 (x8664) GCC version: Could not collect Clang version: 13.1.6 (clang-1316.0.21.2.3) CMake version: version 3.22.1 Libc version: N/A Python version: 3.9.12 (main, Apr 5 2022, 01:53:17) [Clang 12.0.0 ] (64-bit runtime) Python platform: macOS-10.16-x8664-i386-64bit Is CUDA available: N/A CUDA runtime version: Could not collect GPU models and configuration: Could not collect Nvidia driver version: Could not collect cuDNN version: Could not collect HIP runtime version: N/A MIOpen runtime version: N/A Is XNNPACK available: N/A Versions of relevant libraries: [pip3] numpy==1.21.5 [conda] mkl 2022.0.0 hecd8cb5105 [conda] mkl-include 2022.0.0 hecd8cb5105 [conda] numpy 1.21.5 py39h9c3cb841 [conda] numpy-base 1.21.5 py39he782bc11 cc
I am running the same environment and get the same issue. Any insight would be very appreciated
Update: In another repo I get the same error when trying to link to pytorch etc. There I made a minimal case and managed to build when I removed linking to . I can see that we have in the script. Maybe that is the cause of the error?
Looks like they are built with the correct architecture.
More progress, from local minimal case: fails. is just a hello world program.


Question: meant to be installed as pip packages) (default: False). relative_to (str, optional): path of the build file. Required when ``package is True``. It's best to use ``__file__`` for this argument. <del> kwargs: additional arguments that are passed to ffi to declar the </del> <ins> kwargs: additional arguments that are passed to ffi to declare the </ins> extension. See `Extension API reference`_ for details. .. _`Extension API reference`: https://docs.python.org/3/distutils/apiref.html#distutils.core.Extension

If in is removed, then it builds.
The same story with PyTorch 1.10.0. The error appears when I'm trying to build with Apple clang 13.1.6 (Xcode Command Line Tools 13.3). But all works correctly if I build it with Apple clang 13.0 (Xcode Command Line Tools 13.2.1)
Nice, worked for me as well. Is this a bug somewhere or what is the exact problem? I drawback is that XCode needs to be up to date with new iOS versions.
Which version of Apple Clang worked? 13.0.0 or 13.0.1? Are you on Monterey 12.4?
Filed an issue: cc:
This issue has been fixed in PeachPy a while back by but pinned version of PeachPy that PyTorch is using has not been updated in a very long time


Question: { index = z * iwidth * iheight + y * iwidth + x; real val = ip[index]; <del> if (val > maxval) </del> <ins> if ((val > maxval) || isnan(val)) </ins> { maxval = val; maxindex = index;


Question: { index = z * iwidth * iheight + y * iwidth + x; real val = ip[index]; <del> if (val > maxval) </del> <ins> if ((val > maxval) || isnan(val)) </ins> { maxval = val; maxindex = index;


Question: // For Convolution strategies that don't implicitly handle grad_bias, we add a helper // function here to perform it using simple Tensor operators static at::Tensor compute_grad_bias(const at::Tensor& grad_output) { <del> // grad_output is in N, C, H, W, we re-shape and reduce over spatial dims and batches </del> <ins> // grad_output is in N, C, H, W, we re-shape and reduce over spatial dims and batches </ins> return grad_output.contiguous().view({grad_output.size(0), grad_output.size(1), -1}).sum(0).sum(1); }


Question: Args: input (Tensor): the tensor to compare other (Tensor or float): the tensor or value to compare <del> out (Tensor, optional): the output tensor that must be a `ByteTensor` or the same type as :attr:`input` </del> <ins> out (Tensor, optional): the output tensor that must be a `ByteTensor` </ins> Returns: Tensor: A `torch.ByteTensor` containing a 1 at each location where comparison is true

[pytorch] The docs say that for tensor comparison operators (gt,lt etc) it should be possible to pass out argument typed as input (), yet when I try to do it, I hit an error Should the docs be fixed, or is it a bug?
If it's really useful we can add it back; but we'll fix the docs for now


Question: def test_AdaptiveMaxPool3d_indices_cuda(self, dtype=torch.float): self._test_maxpool_indices(3, adaptive=True, device="cuda", dtype=dtype) <ins> @staticmethod def _test_max_pool_nan(self, device, dtype=torch.float): for adaptive in ['', 'adaptive_']: for num_dim in [1, 2, 3]: fn_name = '{}max_pool{}d'.format(adaptive, num_dim) fn = getattr(F, fn_name) x = torch.full([1, 1] + num_dim * [3], float('nan')) res = fn(x, 1 if adaptive else 3) self.assertTrue(math.isnan(res.item())) @unittest.skipIf(not TEST_CUDA, "CUDA unavailable") @repeat_test_for_types(ALL_TENSORTYPES) def test_max_pool_nan_cuda(self, dtype=torch.float): self._test_max_pool_nan(self, device="cuda", dtype=dtype) def test_max_pool_nan(self, dtype=torch.float): self._test_max_pool_nan(self, device="cpu") </ins> def _test_scatter(self, tensor): x = torch.tensor(tensor, requires_grad=True) result = dp.scatter(x, (0, 1))


Question: def test_AdaptiveMaxPool3d_indices_cuda(self, dtype=torch.float): self._test_maxpool_indices(3, adaptive=True, device="cuda", dtype=dtype) <ins> @staticmethod def _test_max_pool_nan(self, device, dtype=torch.float): for adaptive in ['', 'adaptive_']: for num_dim in [1, 2, 3]: fn_name = '{}max_pool{}d'.format(adaptive, num_dim) fn = getattr(F, fn_name) x = torch.full([1, 1] + num_dim * [3], float('nan')) res = fn(x, 1 if adaptive else 3) self.assertTrue(math.isnan(res.item())) @unittest.skipIf(not TEST_CUDA, "CUDA unavailable") @repeat_test_for_types(ALL_TENSORTYPES) def test_max_pool_nan_cuda(self, dtype=torch.float): self._test_max_pool_nan(self, device="cuda", dtype=dtype) def test_max_pool_nan(self, dtype=torch.float): self._test_max_pool_nan(self, device="cpu") </ins> def _test_scatter(self, tensor): x = torch.tensor(tensor, requires_grad=True) result = dp.scatter(x, (0, 1))