Anybody seen this container/ceph-fuse bug?

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

zdude

What is a Computer?
Aug 29, 2017
61
8
8
43
I am trying to mount a ceph-fuse client in a ubuntu 16.04 lxc container within Proxmox v5.1-46. I have gotten far enough that it begins to complain about kernel dentries before failing.

Code:
    -9> 2018-04-08 23:23:36.597140 7ff01595f700  1 client.2979582 using remount_cb
    -8> 2018-04-08 23:23:36.598849 7ff01495d700  3 client.2979582 ll_getattr 0x1.head
    -7> 2018-04-08 23:23:36.598863 7ff01495d700  3 client.2979582 ll_getattr 0x1.head = 0
    -6> 2018-04-08 23:23:36.598876 7ff01495d700  3 client.2979582 ll_forget 0x1 1
    -5> 2018-04-08 23:23:36.598947 7ff01515e700  3 client.2979582 ll_getattr 0x1.head
    -4> 2018-04-08 23:23:36.598957 7ff01515e700  3 client.2979582 ll_getattr 0x1.head = 0
    -3> 2018-04-08 23:23:36.598965 7ff01515e700  3 client.2979582 ll_forget 0x1 1
    -2> 2018-04-08 23:23:36.599479 7ff01595f700 -1 client.2979582 failed to remount (to trim kernel dentries): return code = 32
    -1> 2018-04-08 23:23:36.599497 7ff01595f700 -1 client.2979582 failed to remount for kernel dentry trimming; quitting!
     0> 2018-04-08 23:23:36.600052 7ff01595f700 -1 *** Caught signal (Aborted) **
 in thread 7ff01595f700 thread_name:tester

 ceph version 12.2.4 (52085d5249a80c5f5121a76d6288429f35e4e77b) luminous (stable)
 1: (()+0x6d4784) [0x55a824ac4784]
 2: (()+0x11390) [0x7ff021e2e390]
 3: (gsignal()+0x38) [0x7ff020bba428]
 4: (abort()+0x16a) [0x7ff020bbc02a]
 5: (Client::_do_remount()+0x303) [0x55a824680b83]
 6: (Client::test_dentry_handling(bool)+0x18f) [0x55a824680ebf]
 7: (()+0x20ebe3) [0x55a8245febe3]
 8: (()+0x76ba) [0x7ff021e246ba]
 9: (clone()+0x6d) [0x7ff020c8c41d]
 NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.
I wasn't able to find anything on google, any ideas here? The host has 128GB of ram, 60GB of which is free and the container has 8GB of ram, of which all but 70MB is free. (specified because alot of these errors appear to come form out of memory issues)

UPDATE: I can verify it is not an issue with the fuse inside the container. I was able to use sshfs to fuse mount a directory without issue.


New
UPDATE: 1 Year in

Long time since I updated this one... Sometime in the past year it has become possible to mount ceph-fuse clients without any code changes to ceph or proxmox. Under the options tab of the client edit features to have both nesting and fuse enabled. Once this is done ceph-fuse will work as in a VM or metal
 
Last edited:

Patrick

Administrator
Staff member
Dec 21, 2010
12,511
5,792
113
I have one for ZFS rpool and Docker (not supported by Proxmox) that I documented the fix for, but I have not seen this one.
 

zdude

What is a Computer?
Aug 29, 2017
61
8
8
43
Do you have a link to the fix you have documented? At this point I am reading into stuff that is only tangentially related to try finding a fix...
 

zdude

What is a Computer?
Aug 29, 2017
61
8
8
43
Small update, I am diving into the ceph source code and compiling with debug messages in the troublesome methods. This is what I have gotten so far before the fuse client crashes.

Code:
2018-04-19 22:45:25.544 7fecf9371c00  0 ceph version 13.0.2-1552-ga4778e9 (a4778e9d62fd38eb1e4eba1cfb3bc27a2ae5e101) mimic (dev), process (unknown), pid 581164
2018-04-19 22:45:25.544 7fecf9371c00 -1 init, newargv = 0x560d9424e760 newargc=7
ceph-fuse[581164]: starting ceph client
ceph-fuse[581164]: starting fuse
mount: cannot remount /home/super/ceph/build/bin/ceph-fuse read-write, is write-protected
I have narrowed it down to a call to libfuse that is failing at src/client/fuse_ll.cc:1203 Not sure what is causing the failure at this time, however I will continue plugging away at the issue when I have time.
 

zdude

What is a Computer?
Aug 29, 2017
61
8
8
43
Long time since I updated this one... Sometime in the past year it has become possible to mount ceph-fuse clients without any code changes to ceph or proxmox. Under the options tab of the client edit features to have both nesting and fuse enabled. Once this is done ceph-fuse will work as in a VM or metal.