Ytukyg

After setting up a docker swarm with cloudformation using the info from https://docs.docker.com/docker-for-aws/. The swarm runs a Django stack (django, nginx, celery) and for media storage a docker EFS volume is used (same setup as the shared storage volume in https://docs.docker.com/docker-for-aws/persistent-data-volumes/).

When I'm uploading a file through the website (using a django FileField) the django container hangs and in the corresponding EC2 instance I see the following kernel errors:

Nov 25 22:05:47 moby kernel: INFO: task uwsgi:6951 blocked for more than 120 seconds.

Nov 25 22:05:47 moby kernel:       Not tainted 4.9.114-moby #1

Nov 25 22:05:47 moby kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.

Nov 25 22:05:47 moby kernel: uwsgi           D    0  6951   6504 0x00000100

Nov 25 22:05:47 moby kernel:  00000000000190c0 0000000000000000 ffff9a00a6e9a400 ffff9a009ee5c340

Nov 25 22:05:47 moby kernel:  ffff9a00b2e80d00 ffff9a00b6e190c0 ffffffffb583caf6 0000000000000002

Nov 25 22:05:47 moby kernel:  ffff9a00b2e80d00 ffffa767c0bdfce0 7fffffffffffffff 0000000000000002

Nov 25 22:05:47 moby kernel: Call Trace:

Nov 25 22:05:47 moby kernel:  [<ffffffffb583caf6>] ? __schedule+0x35f/0x43d

Nov 25 22:05:47 moby kernel:  [<ffffffffb583cf26>] ? bit_wait+0x2a/0x2a

Nov 25 22:05:47 moby kernel:  [<ffffffffb583cc52>] ? schedule+0x7e/0x87

Nov 25 22:05:47 moby kernel:  [<ffffffffb583e8de>] ? schedule_timeout+0x43/0x101

Nov 25 22:05:47 moby kernel:  [<ffffffffb5019808>] ? xen_clocksource_read+0x11/0x12

Nov 25 22:05:47 moby kernel:  [<ffffffffb512e281>] ? timekeeping_get_ns+0x19/0x2c

Nov 25 22:05:47 moby kernel:  [<ffffffffb583c739>] ? io_schedule_timeout+0x99/0xf7

Nov 25 22:05:47 moby kernel:  [<ffffffffb583c739>] ? io_schedule_timeout+0x99/0xf7

Nov 25 22:05:47 moby kernel:  [<ffffffffb583cf3d>] ? bit_wait_io+0x17/0x34

Nov 25 22:05:47 moby kernel:  [<ffffffffb583d009>] ? __wait_on_bit+0x48/0x76

Nov 25 22:05:47 moby kernel:  [<ffffffffb519e758>] ? wait_on_page_bit+0x7c/0x96

Nov 25 22:05:47 moby kernel:  [<ffffffffb510f99e>] ? autoremove_wake_function+0x35/0x35

Nov 25 22:05:47 moby kernel:  [<ffffffffb519e842>] ? __filemap_fdatawait_range+0xd0/0x12b

Nov 25 22:05:47 moby kernel:  [<ffffffffb519e8ac>] ? filemap_fdatawait_range+0xf/0x23

Nov 25 22:05:47 moby kernel:  [<ffffffffb51a060c>] ? filemap_write_and_wait_range+0x3a/0x4f

Nov 25 22:05:47 moby kernel:  [<ffffffffb52bcf98>] ? nfs_file_fsync+0x54/0x187

Nov 25 22:05:47 moby kernel:  [<ffffffffb51f6c4d>] ? filp_close+0x39/0x66

Nov 25 22:05:47 moby kernel:  [<ffffffffb51f6c99>] ? SyS_close+0x1f/0x47

Nov 25 22:05:47 moby kernel:  [<ffffffffb50033b7>] ? do_syscall_64+0x69/0x79

Nov 25 22:05:47 moby kernel:  [<ffffffffb583f64e>] ? entry_SYSCALL_64_after_swapgs+0x58/0xc6

Nov 25 22:06:28 moby kernel: nfs: server fs-d5eb171d.efs.eu-west-1.amazonaws.com not responding, still trying

I could resort to using S3 buckets, but this will take quite an effort. Is there something obvious that I'm missing/doing wrong? Or do I need to take an alternative route for storage?

Apparently I needed to downgrade from Docker 18.06 to 18.03

Now it all works :)