RuntimeError: DataLoader worker is killed by signal: Illegal instruction.
1
I am using the following code. I run this on a GPU machine with below config: nvcc: NVIDIA (R) Cuda compiler driver Copyright (c) 2005-2017 NVIDIA Corporation Built on Fri_Sep__1_21:08:03_CDT_2017 Cuda compilation tools, release 9.0, V9.0.176 I am using pytorch version 1.0.0.dev20181123. I face this error: RuntimeError: DataLoader worker (pid 23646) is killed by signal: Illegal instruction. Thinking that its a shared memory problem, I tried the solution here at: https://www.lucidarme.me/increase-shared-memory-limit/ It didn't help. Any pointers? import argparse import json import os import torch #=====START: ADDED FOR DISTRIBUTED====== from distributed import init_distributed, apply_gradient_allreduce, reduce_tensor from torch.utils.data.distributed import DistributedSampler #=====END:...