Need help: Docker, ZFS & SAP HANA

Discussion in 'Docker and Containers' started by nthu9280, Oct 30, 2018.

  1. nthu9280

    nthu9280 Well-Known Member

    Joined:
    Feb 3, 2016
    Messages:
    1,092
    Likes Received:
    250
    zfs experts -
    I have a Ubuntu 18.04.1 LTS host with ZFS hosting /var/lib/docker and it generally works fine. I did experience some quirks but that can be another topic.

    I'm trying to setup SAP HANA express edition docker and ran into an issue. It's not a supported config by SAP. But someone has posted a work around but that is not quite working for me.

    Installing SAP HANA, express edition with Docker - persistence on zfs · Issue #2234 · SAPDocuments/Tutorials

    I created a 200G image as described above and mounted but still getting the error.


    Code:
    dd if=/dev/zero of=HANA.img bs=128k seek=1638400 count=0
    mkfs.ext4 HANA.img
    mount HANA.img /mnt/hxe1data
    chown hxeadm.sapsys /mnt/hxe1data
    chmod 666 /mnt/hxe1data
    
    I'm using the latest release if that makes a difference
    store/saplabs/hanaexpressxsa:2.00.033.00.20180925.2
    The base OS is Ubuntu 18.04.1 LTS
    Docker -
    Server Version: 18.06.1-ce
    
    Snippet of error log:
    
    Change hardware key...
    nameserver hxehost:39001 not responding.
    Opening persistence ...
    assign failed with persistence startup error. exception 1: no.2000004 (Basis/IO/FileAccess/impl/FileDescriptors.cpp:334)
    Cannot open file ""/datavolume_0000.dat" ((closed, mode= RW, access= rw-rw-r--, flags= DIRECT|MUST_EXIST|LOCK), factory= (root= "/hana/shared/data/HXE/mnt00001/hdb00001/" (access= rw-rw-r--, flags= AUTOCREATE_DIRECTORY, usage= DATA, fs= UNKNOWN (zfs), config= (async_write_submit_active=on,async_write_submit_blocks=all,async_read_submit=on,num_submit_queues=1,num_completion_queues=1,size_kernel_io_queue=512,max_parallel_io_requests=64,min_submit_batch_size=16,max_submit_batch_size=64))", rc=22: Invalid argument
    exception throw location:
    
     stopping service...
    error: Unknown persistence layer error
    failed. trace file nameserver_hxehost.00000.000.trc may contain more error details.
    ~~~
    
    *****
     
    #1
  2. Evan

    Evan Well-Known Member

    Joined:
    Jan 6, 2016
    Messages:
    2,309
    Likes Received:
    322
    I had no idea such a toy was available for HANA, cool. Sorry can’t help to fix that error though.
     
    #2
    MiniKnight likes this.
  3. nthu9280

    nthu9280 Well-Known Member

    Joined:
    Feb 3, 2016
    Messages:
    1,092
    Likes Received:
    250
    Yes, HANA Express & HANA Express XSA are both free upto 32GB license even for production use.

    Developer license has higher limit 64gb memory instance but the hassle of renewing every 90 days and can't be used for production purpose.

    I think I'm close to get the docker version working. HANA doesn't like ZFS. Just need to present the volume for persistence as described above in a supported fs such as ext4. Don't know what I'm doing wrong.
     
    #3
  4. gigatexal

    gigatexal I'm here to learn

    Joined:
    Nov 25, 2012
    Messages:
    2,485
    Likes Received:
    440
    For testing maybe try chmod 777 on the file for now. Perhaps the docker user doesn’t have perms to the file you created?
     
    #4
    Last edited: Oct 31, 2018
  5. nthu9280

    nthu9280 Well-Known Member

    Joined:
    Feb 3, 2016
    Messages:
    1,092
    Likes Received:
    250
    I've already tried that.
     
    #5
  6. gigatexal

    gigatexal I'm here to learn

    Joined:
    Nov 25, 2012
    Messages:
    2,485
    Likes Received:
    440
    Hmm... maybe the mailing list if there is one ?
     
    #6
  7. marcoi

    marcoi Active Member

    Joined:
    Apr 6, 2013
    Messages:
    970
    Likes Received:
    154
    did you do the basic troubleshooting steps? Sometimes the obvious is easily overlooked.

    Check the storage area? IE can you write a file, read it, edit it, etc.?
    Can you temporary unmount the zfs storage and use local disk to confirm if its a ZFS issue vs container issue?
     
    #7
  8. nthu9280

    nthu9280 Well-Known Member

    Joined:
    Feb 3, 2016
    Messages:
    1,092
    Likes Received:
    250
    For now, I'll shelve the docker approach and use a VM instead.

    I'm presenting ext4 FS (/mnt/hxe1data) for volume but the install inside the docker still sees it as ZFS and don't know how to work with it.
    -v /mnt/hxe1data:/hana/mounts \
    ~~
    Cannot open file ""<root>/datavolume_0000.dat" ((closed, mode= RW, access= rw-rw-r--, flags= DIRECT|MUST_EXIST|LOCK), factory= (root= "/hana/shared/data/HXE/mnt00001/hdb00001/" (access= rw-rw-r--, flags= AUTOCREATE_DIRECTORY, usage= DATA, fs= UNKNOWN (zfs)
    ~~
     
    #8
  9. marcoi

    marcoi Active Member

    Joined:
    Apr 6, 2013
    Messages:
    970
    Likes Received:
    154
    Maybe your missing something at the host level that tells dockers what kind of mount it suppose to be?
    IDK if related but maybe how you configure the mount and using the /etc/fstab?

    Maybe you need to configure the mount in fstab with different settings.

    Currently im using freeNas and NFS mount to my ubuntu images using the below line in fstab
    dev-freenas:/mnt/Storage/ /Software/ nfs auto,noatime,nolock,bg,nfsvers=3,intr,tcp,actimeo=1800 0 0

    I'm not an expert on fstab and not sure if you are using that or not is your issue but I figured i mention it.
     
    #9
  10. nthu9280

    nthu9280 Well-Known Member

    Joined:
    Feb 3, 2016
    Messages:
    1,092
    Likes Received:
    250
    Guess I can mount another drive and format ext4 and present that for volume for testing. However, /var/lib/docker is on ZFS. Someone has done this as described in the link I posted in my OP. So like you said I'm probably overlooking something simple

    I've done basic testing of creating a test file and deleting it etc.

    Code:
    root@ubnt-x10sra1:/home/nthu9280# su hxeadm
    hxeadm@ubnt-x10sra1:/home/nthu9280$ cd /mnt/hxe1data/
    hxeadm@ubnt-x10sra1:/mnt/hxe1data$ ls
    lost+found
    hxeadm@ubnt-x10sra1:/mnt/hxe1data$ touch test.txt
    hxeadm@ubnt-x10sra1:/mnt/hxe1data$ ls
    lost+found  test.txt
    hxeadm@ubnt-x10sra1:/mnt/hxe1data$ rm touch.txt
    rm: cannot remove 'touch.txt': No such file or directory
    hxeadm@ubnt-x10sra1:/mnt/hxe1data$ rm test.txt
    hxeadm@ubnt-x10sra1:/mnt/hxe1data$ ls
    lost+found
    hxeadm@ubnt-x10sra1:/mnt/hxe1data$ cd ..
    hxeadm@ubnt-x10sra1:/mnt$ ls -l
    total 12
    drwxr-xr-x 3 root   root   4096 Oct 29 19:45 disk1
    drwxr-xr-x 3 root   root   4096 Oct 30 09:45 hana1
    drwxrwxrwx 3 hxeadm sapsys 4096 Oct 31 11:48 hxe1data
    hxeadm@ubnt-x10sra1:/mnt$
    
     
    #10
  11. marcoi

    marcoi Active Member

    Joined:
    Apr 6, 2013
    Messages:
    970
    Likes Received:
    154
    does an application like gparted show zfs mounts devices? If so maybe you can check to confirm it formated correctly as well?
     
    #11
  12. nthu9280

    nthu9280 Well-Known Member

    Joined:
    Feb 3, 2016
    Messages:
    1,092
    Likes Received:
    250
    I followed this to configure Docker to use ZFS.

    Use the ZFS storage driver

    It works ok for other test containers. I'm playing with Hortonworks HDP sandbox etc.

    I do have a issue but that is documented.

    ~~
    Storage Driver: zfs
    Zpool: error while getting pool information strconv.ParseUint: parsing "": invalid syntax
    Zpool Health: not available
    ~~

    Code:
    # docker info
    Containers: 4
     Running: 0
     Paused: 0
     Stopped: 4
    Images: 85
    Server Version: 18.06.1-ce
    Storage Driver: zfs
     Zpool: error while getting pool information strconv.ParseUint: parsing "": invalid syntax
     Zpool Health: not available
     Parent Dataset: zpl2/docker
     Space Used By Parent: 256438519920
     Space Available: 5491961909104
     Parent Quota: no
     Compression: off
    Logging Driver: json-file
    Cgroup Driver: cgroupfs
    Plugins:
     Volume: local
     Network: bridge host macvlan null overlay
     Log: awslogs fluentd gcplogs gelf journald json-file logentries splunk syslog
    Swarm: inactive
    Runtimes: runc nvidia
    Default Runtime: runc
    Init Binary: docker-init
    containerd version: 468a545b9edcd5932818eb9de8e72413e616e86e
    runc version: 69663f0bd4b60df09991c08812a60108003fa340
    init version: fec3683
    Security Options:
     apparmor
     seccomp
      Profile: default
    Kernel Version: 4.15.0-38-generic
    Operating System: Ubuntu 18.04.1 LTS
    OSType: linux
    Architecture: x86_64
    CPUs: 8
    Total Memory: 220.3GiB
    Name: ubnt-x10sra1
    ID: VLTZ:ETKS:DO4D:6IZP:MHRW:3SQZ:FY72:U5JC:YZOJ:WAZV:YF2D:EVRF
    Docker Root Dir: /var/lib/docker
    Debug Mode (client): false
    Debug Mode (server): false
    Username: nthu9280
    Registry: https://index.docker.io/v1/
    Labels:
    Experimental: false
    Insecure Registries:
     127.0.0.0/8
    Live Restore Enabled: false
    
    WARNING: No swap limit support
    
    
     
    #12
Similar Threads: Need help
Forum Title Date
Docker and Containers nvidia-docker on Ubuntu 16.04 and need to change docker installation method May 12, 2017

Share This Page