Uploaded image for project: 'openATTIC'
  1. openATTIC
  2. OP-2045

Creating a volume on an erasure coded or replicated coded Ceph pool fails

    XMLWordPrintable

    Details

      Description

      The creation of a volume on a erasure coded pool produces errors in the log.

      The following will appear in the log.

      From the opneattic log:

      2017-03-23 15:25:47,816 - INFO - openattic_systemd#loggedfunc - Calling start_queue()
      2017-03-23 15:25:47,821 - INFO - openattic_systemd#loggedfunc - Calling acquire_lock(dbus.String(u'/var/lock/openattic/volume-cda2ae26-db63-4082-9ab0-09696afb251c'))
      2017-03-23 15:25:47,839 - INFO - openattic_systemd#loggedfunc - Calling /nagios::writeconf()
      2017-03-23 15:25:47,839 - INFO - openattic_systemd#wrapper - [27005/:1.13524] Deferring call to /nagios::writeconf()
      2017-03-23 15:25:47,840 - INFO - openattic_systemd#loggedfunc - Calling /nagios::restart_service()
      2017-03-23 15:25:47,840 - INFO - openattic_systemd#wrapper - [27005/:1.13524] Deferring call to /nagios::restart_service()
      2017-03-23 15:25:47,844 - INFO - openattic_systemd#loggedfunc - Calling /ceph::rbd_create(dbus.String(u'ceph'), dbus.String(u'epool'), dbus.String(u'elun'), dbus.Int32(100))
      2017-03-23 15:25:47,844 - INFO - openattic_systemd#wrapper - [27005/:1.13524] Deferring call to /ceph::rbd_create(dbus.String(u'ceph'), dbus.String(u'epool'), dbus.String(u'elun'), dbus.Int32(100))
      2017-03-23 15:25:47,845 - INFO - openattic_systemd#loggedfunc - Calling /ceph::rbd_map(dbus.String(u'ceph'), dbus.String(u'epool'), dbus.String(u'elun'))
      2017-03-23 15:25:47,846 - INFO - openattic_systemd#wrapper - [27005/:1.13524] Deferring call to /ceph::rbd_map(dbus.String(u'ceph'), dbus.String(u'epool'), dbus.String(u'elun'))
      2017-03-23 15:25:47,847 - INFO - openattic_systemd#loggedfunc - Calling run_queue_background()
      2017-03-23 15:25:47,850 - INFO - openattic_systemd#_run_queue - [29561/:1.13524] Incoming Queue Dump:
      2017-03-23 15:25:47,851 - INFO - openattic_systemd#_run_queue - [29561/:1.13524] -> /nagios::writeconf()
      2017-03-23 15:25:47,851 - INFO - openattic_systemd#_run_queue - [29561/:1.13524] -> /nagios::restart_service()
      2017-03-23 15:25:47,851 - INFO - openattic_systemd#_run_queue - [29561/:1.13524] -> /ceph::rbd_create(dbus.String(u'ceph'), dbus.String(u'epool'), dbus.String(u'elun'), dbus.Int32(100))
      2017-03-23 15:25:47,851 - INFO - openattic_systemd#_run_queue - [29561/:1.13524] -> /ceph::rbd_map(dbus.String(u'ceph'), dbus.String(u'epool'), dbus.String(u'elun'))
      2017-03-23 15:25:47,852 - INFO - openattic_systemd#_run_queue - [29561/:1.13524] End of queue dump.
      2017-03-23 15:25:47,852 - INFO - openattic_systemd#_run_queue - [29561/:1.13524] Acquiring lock '/var/lock/openattic/volume-cda2ae26-db63-4082-9ab0-09696afb251c'...
      2017-03-23 15:25:47,853 - INFO - openattic_systemd#_run_queue - [29561/:1.13524] Acquired lock '/var/lock/openattic/volume-cda2ae26-db63-4082-9ab0-09696afb251c'.
      2017-03-23 15:25:47,853 - INFO - openattic_systemd#_run_queue - [29561/:1.13524] Executing deferred call to /nagios::writeconf()
      2017-03-23 15:25:47,868 - ERROR - root#acquire_lock - /var/lock/openattic/volume-cda2ae26-db63-4082-9ab0-09696afb251c has been locked for more than 0 seconds (PID 29561)
      2017-03-23 15:25:47,891 - INFO - openattic_systemd#_run_queue - [29561/:1.13524] Executing deferred call to /nagios::restart_service()
      2017-03-23 15:25:47,904 - INFO - openattic_systemd#service_command - service_command(nagios3): invoking `service nagios3 reload`
      2017-03-23 15:25:47,952 - INFO - openattic_systemd#_run_queue - [29561/:1.13524] Executing deferred call to /ceph::rbd_create(dbus.String(u'ceph'), dbus.String(u'epool'), dbus.String(u'elun'), dbus.Int32(100))
      2017-03-23 15:25:48,052 - ERROR - root#acquire_lock - /var/lock/openattic/volume-cda2ae26-db63-4082-9ab0-09696afb251c has been locked for more than 0 seconds (PID 29561)
      2017-03-23 15:25:48,089 - INFO - openattic_systemd#loggedfunc - Calling /volumes::fs_stat(dbus.String(u'/media/tank'))
      2017-03-23 15:25:49,101 - ERROR - openattic_systemd#invoke - > "rbd" "-c" "/etc/ceph/ceph.conf" "-p" "epool" "create" "elun" "--size" "100"
      E 2017-03-23 15:25:49.092738 7f3fb836cd80 -1 librbd: error adding image to directory: (95) Operation not supported
      E rbd: create error: (95) Operation not supported
      2017-03-23 15:25:49,105 - ERROR - openattic_systemd#_run_queue - Received error:
      Traceback (most recent call last):
      File "/srv/openattic/backend/systemd/management/commands/runsystemd.py", line 117, in _run_queue
      func(scope, *args, **kwargs)
      File "/srv/openattic/backend/ceph/systemapi.py", line 110, in rbd_create
      str(megs)])
      File "/srv/openattic/backend/systemd/procutils.py", line 116, in invoke
      raise SystemError("%s failed: %s" % (cmdline, procerr))
      SystemError: "rbd" "-c" "/etc/ceph/ceph.conf" "-p" "epool" "create" "elun" "--size" "100" failed: 2017-03-23 15:25:49.092738 7f3fb836cd80 -1 librbd: error adding image to directory: (95) Operation not supported
      rbd: create error: (95) Operation not supported
      

      After sometime the volume will be accessible and be mounted under /dev/rbd/<poolName>/<volname> if it is a lun if it is anything else it will be mounted under /media/<volume>

      But no rbd will be found under ceph rbds like when I'm creating volumes on a replicated pool.

      But the replicated pool will also throw an error in a task:

      2017-03-23 16:10:31,096 - INFO - openattic_systemd#loggedfunc - Calling /lvm::vgs()
      2017-03-23 16:10:31,139 - INFO - openattic_systemd#loggedfunc - Calling /lvm::vgs()
      2017-03-23 16:10:31,159 - INFO - openattic_systemd#loggedfunc - Calling /zfs::zfs_get(dbus.String(u'tank'), dbus.String(u'used'))
      2017-03-23 16:10:31,170 - INFO - openattic_systemd#loggedfunc - Calling /zfs::zfs_get(dbus.String(u'tank'), dbus.String(u'available'))
      2017-03-23 16:10:31,175 - INFO - openattic_systemd#loggedfunc - Calling /zfs::zfs_get(dbus.String(u'tank'), dbus.String(u'available'))
      2017-03-23 16:10:31,187 - INFO - openattic_systemd#loggedfunc - Calling /zfs::zpool_get(dbus.String(u'tank'), dbus.String(u'health'))
      2017-03-23 16:10:31,197 - INFO - openattic_systemd#loggedfunc - Calling /zfs::zpool_get(dbus.String(u'tank'), dbus.String(u'health'))
      2017-03-23 16:10:31,207 - INFO - openattic_systemd#loggedfunc - Calling /volumes::fs_stat(dbus.String(u'/media/tank'))
      2017-03-23 16:10:31,220 - INFO - openattic_systemd#loggedfunc - Calling /ceph::df(dbus.String(u'ceph'))
      2017-03-23 16:10:31,375 - INFO - openattic_systemd#loggedfunc - Calling /ceph::status(dbus.String(u'ceph'))
      2017-03-23 16:10:31,539 - INFO - openattic_systemd#loggedfunc - Calling /ceph::df(dbus.String(u'ceph'))
      2017-03-23 16:10:31,548 - INFO - openattic_systemd#loggedfunc - Calling /ceph::status(dbus.String(u'ceph'))
      2017-03-23 16:10:31,561 - INFO - openattic_systemd#loggedfunc - Calling /ceph::df(dbus.String(u'ceph'))
      2017-03-23 16:10:31,569 - INFO - openattic_systemd#loggedfunc - Calling /ceph::status(dbus.String(u'ceph'))
      2017-03-23 16:10:48,925 - INFO - openattic_systemd#loggedfunc - Calling start_queue()
      2017-03-23 16:10:48,930 - INFO - openattic_systemd#loggedfunc - Calling acquire_lock(dbus.String(u'/var/lock/openattic/volume-df840a51-80e7-4aaa-9a26-073d1050a0ad'))
      2017-03-23 16:10:48,948 - INFO - openattic_systemd#loggedfunc - Calling /nagios::writeconf()
      2017-03-23 16:10:48,948 - INFO - openattic_systemd#wrapper - [27005/:1.13562] Deferring call to /nagios::writeconf()
      2017-03-23 16:10:48,948 - INFO - openattic_systemd#loggedfunc - Calling /nagios::restart_service()
      2017-03-23 16:10:48,949 - INFO - openattic_systemd#wrapper - [27005/:1.13562] Deferring call to /nagios::restart_service()
      2017-03-23 16:10:48,952 - INFO - openattic_systemd#loggedfunc - Calling /ceph::rbd_create(dbus.String(u'ceph'), dbus.String(u'rbd'), dbus.String(u'rlun'), dbus.Int32(100))
      2017-03-23 16:10:48,952 - INFO - openattic_systemd#wrapper - [27005/:1.13562] Deferring call to /ceph::rbd_create(dbus.String(u'ceph'), dbus.String(u'rbd'), dbus.String(u'rlun'), dbus.Int32(100))
      2017-03-23 16:10:48,954 - INFO - openattic_systemd#loggedfunc - Calling /ceph::rbd_map(dbus.String(u'ceph'), dbus.String(u'rbd'), dbus.String(u'rlun'))
      2017-03-23 16:10:48,954 - INFO - openattic_systemd#wrapper - [27005/:1.13562] Deferring call to /ceph::rbd_map(dbus.String(u'ceph'), dbus.String(u'rbd'), dbus.String(u'rlun'))
      2017-03-23 16:10:48,955 - INFO - openattic_systemd#loggedfunc - Calling run_queue_background()
      2017-03-23 16:10:48,959 - INFO - openattic_systemd#_run_queue - [4832/:1.13562] Incoming Queue Dump:
      2017-03-23 16:10:48,959 - INFO - openattic_systemd#_run_queue - [4832/:1.13562] -> /nagios::writeconf()
      2017-03-23 16:10:48,960 - INFO - openattic_systemd#_run_queue - [4832/:1.13562] -> /nagios::restart_service()
      2017-03-23 16:10:48,960 - INFO - openattic_systemd#_run_queue - [4832/:1.13562] -> /ceph::rbd_create(dbus.String(u'ceph'), dbus.String(u'rbd'), dbus.String(u'rlun'), dbus.Int32(100))
      2017-03-23 16:10:48,960 - INFO - openattic_systemd#_run_queue - [4832/:1.13562] -> /ceph::rbd_map(dbus.String(u'ceph'), dbus.String(u'rbd'), dbus.String(u'rlun'))
      2017-03-23 16:10:48,960 - INFO - openattic_systemd#_run_queue - [4832/:1.13562] End of queue dump.
      2017-03-23 16:10:48,961 - INFO - openattic_systemd#_run_queue - [4832/:1.13562] Acquiring lock '/var/lock/openattic/volume-df840a51-80e7-4aaa-9a26-073d1050a0ad'...
      2017-03-23 16:10:48,961 - INFO - openattic_systemd#_run_queue - [4832/:1.13562] Acquired lock '/var/lock/openattic/volume-df840a51-80e7-4aaa-9a26-073d1050a0ad'.
      2017-03-23 16:10:48,961 - INFO - openattic_systemd#_run_queue - [4832/:1.13562] Executing deferred call to /nagios::writeconf()
      2017-03-23 16:10:48,965 - ERROR - root#acquire_lock - /var/lock/openattic/volume-df840a51-80e7-4aaa-9a26-073d1050a0ad has been locked for more than 0 seconds (PID 4832)
      2017-03-23 16:10:49,021 - INFO - openattic_systemd#_run_queue - [4832/:1.13562] Executing deferred call to /nagios::restart_service()
      2017-03-23 16:10:49,038 - INFO - openattic_systemd#service_command - service_command(nagios3): invoking `service nagios3 reload`
      2017-03-23 16:10:49,097 - INFO - openattic_systemd#_run_queue - [4832/:1.13562] Executing deferred call to /ceph::rbd_create(dbus.String(u'ceph'), dbus.String(u'rbd'), dbus.String(u'rlun'), dbus.Int32(100))
      2017-03-23 16:10:49,155 - INFO - openattic_systemd#_run_queue - [4832/:1.13562] Executing deferred call to /ceph::rbd_map(dbus.String(u'ceph'), dbus.String(u'rbd'), dbus.String(u'rlun'))
      2017-03-23 16:10:49,162 - INFO - openattic_systemd#loggedfunc - Calling /volumes::fs_stat(dbus.String(u'/media/eeext4'))
      2017-03-23 16:10:49,170 - INFO - openattic_systemd#loggedfunc - Calling /ceph::status(dbus.String(u'ceph'))
      2017-03-23 16:10:49,347 - INFO - openattic_systemd#loggedfunc - Calling /volumes::fs_stat(dbus.String(u'/media/eeext4'))
      2017-03-23 16:10:49,376 - INFO - openattic_systemd#loggedfunc - Calling /volumes::fs_stat(dbus.String(u'/media/eext4'))
      2017-03-23 16:10:49,384 - INFO - openattic_systemd#loggedfunc - Calling /ceph::status(dbus.String(u'ceph'))
      2017-03-23 16:10:49,391 - INFO - openattic_systemd#loggedfunc - Calling /volumes::fs_stat(dbus.String(u'/media/eext4'))
      2017-03-23 16:10:49,421 - INFO - openattic_systemd#loggedfunc - Calling /ceph::status(dbus.String(u'ceph'))
      2017-03-23 16:10:49,445 - INFO - openattic_systemd#loggedfunc - Calling /volumes::fs_stat(dbus.String(u'/media/exfs'))
      2017-03-23 16:10:49,451 - INFO - openattic_systemd#loggedfunc - Calling /ceph::status(dbus.String(u'ceph'))
      2017-03-23 16:10:49,458 - INFO - openattic_systemd#loggedfunc - Calling /volumes::fs_stat(dbus.String(u'/media/exfs'))
      2017-03-23 16:10:49,482 - ERROR - root#acquire_lock - /var/lock/openattic/volume-df840a51-80e7-4aaa-9a26-073d1050a0ad has been locked for more than 0 seconds (PID 4832)
      2017-03-23 16:10:49,503 - INFO - openattic_systemd#loggedfunc - Calling /volumes::fs_stat(dbus.String(u'/media/tank'))
      2017-03-23 16:11:01,636 - INFO - openattic_systemd#loggedfunc - Calling /volumes::fs_stat(dbus.String(u'/media/eeext4'))
      2017-03-23 16:11:01,680 - INFO - openattic_systemd#loggedfunc - Calling /volumes::fs_stat(dbus.String(u'/media/eext4'))
      2017-03-23 16:11:01,724 - INFO - openattic_systemd#loggedfunc - Calling /volumes::fs_stat(dbus.String(u'/media/exfs'))
      2017-03-23 16:11:01,748 - ERROR - root#acquire_lock - /var/lock/openattic/volume-df840a51-80e7-4aaa-9a26-073d1050a0ad has been locked for more than 0 seconds (PID 4832)
      2017-03-23 16:11:01,770 - INFO - openattic_systemd#loggedfunc - Calling /volumes::fs_stat(dbus.String(u'/media/tank'))
      2017-03-23 16:11:04,039 - INFO - openattic_systemd#loggedfunc - Calling /volumes::fs_stat(dbus.String(u'/media/eeext4'))
      2017-03-23 16:11:04,085 - INFO - openattic_systemd#loggedfunc - Calling /volumes::fs_stat(dbus.String(u'/media/eext4'))
      2017-03-23 16:11:04,130 - INFO - openattic_systemd#loggedfunc - Calling /volumes::fs_stat(dbus.String(u'/media/exfs'))
      2017-03-23 16:11:04,154 - ERROR - root#acquire_lock - /var/lock/openattic/volume-df840a51-80e7-4aaa-9a26-073d1050a0ad has been locked for more than 0 seconds (PID 4832)
      2017-03-23 16:11:04,177 - INFO - openattic_systemd#loggedfunc - Calling /volumes::fs_stat(dbus.String(u'/media/tank'))
      2017-03-23 16:11:10,768 - INFO - openattic_systemd#loggedfunc - Calling /lvm::vgs()
      2017-03-23 16:11:10,813 - INFO - openattic_systemd#loggedfunc - Calling /lvm::vgs()
      2017-03-23 16:11:10,828 - INFO - openattic_systemd#loggedfunc - Calling /zfs::zfs_get(dbus.String(u'tank'), dbus.String(u'used'))
      2017-03-23 16:11:10,838 - INFO - openattic_systemd#loggedfunc - Calling /zfs::zfs_get(dbus.String(u'tank'), dbus.String(u'available'))
      2017-03-23 16:11:10,843 - INFO - openattic_systemd#loggedfunc - Calling /zfs::zfs_get(dbus.String(u'tank'), dbus.String(u'available'))
      2017-03-23 16:11:10,860 - INFO - openattic_systemd#loggedfunc - Calling /ceph::df(dbus.String(u'ceph'))
      2017-03-23 16:11:11,027 - INFO - openattic_systemd#loggedfunc - Calling /ceph::df(dbus.String(u'ceph'))
      2017-03-23 16:11:11,039 - INFO - openattic_systemd#loggedfunc - Calling /ceph::df(dbus.String(u'ceph'))
      2017-03-23 16:11:26,917 - INFO - openattic_systemd#loggedfunc - Calling start_queue()
      2017-03-23 16:11:26,922 - INFO - openattic_systemd#loggedfunc - Calling acquire_lock(dbus.String(u'/var/lock/openattic/volume-142baf21-8f24-414b-9aa4-b4a2bd56c2f5'))
      2017-03-23 16:11:26,936 - INFO - openattic_systemd#loggedfunc - Calling /nagios::writeconf()
      2017-03-23 16:11:26,936 - INFO - openattic_systemd#wrapper - [27005/:1.13534] Deferring call to /nagios::writeconf()
      2017-03-23 16:11:26,937 - INFO - openattic_systemd#loggedfunc - Calling /nagios::restart_service()
      2017-03-23 16:11:26,937 - INFO - openattic_systemd#wrapper - [27005/:1.13534] Deferring call to /nagios::restart_service()
      2017-03-23 16:11:26,941 - INFO - openattic_systemd#loggedfunc - Calling /ceph::rbd_create(dbus.String(u'ceph'), dbus.String(u'rbd'), dbus.String(u'rext4'), dbus.Int32(100))
      2017-03-23 16:11:26,941 - INFO - openattic_systemd#wrapper - [27005/:1.13534] Deferring call to /ceph::rbd_create(dbus.String(u'ceph'), dbus.String(u'rbd'), dbus.String(u'rext4'), dbus.Int32(100))
      2017-03-23 16:11:26,943 - INFO - openattic_systemd#loggedfunc - Calling /ceph::rbd_map(dbus.String(u'ceph'), dbus.String(u'rbd'), dbus.String(u'rext4'))
      2017-03-23 16:11:26,943 - INFO - openattic_systemd#wrapper - [27005/:1.13534] Deferring call to /ceph::rbd_map(dbus.String(u'ceph'), dbus.String(u'rbd'), dbus.String(u'rext4'))
      2017-03-23 16:11:26,959 - INFO - openattic_systemd#loggedfunc - Calling /nagios::writeconf()
      2017-03-23 16:11:26,959 - INFO - openattic_systemd#wrapper - [27005/:1.13534] Deferring call to /nagios::writeconf()
      2017-03-23 16:11:26,960 - INFO - openattic_systemd#loggedfunc - Calling /nagios::restart_service()
      2017-03-23 16:11:26,960 - INFO - openattic_systemd#wrapper - [27005/:1.13534] Deferring call to /nagios::restart_service()
      2017-03-23 16:11:26,964 - INFO - openattic_systemd#loggedfunc - Calling /volumes::e4fs_format(dbus.String(u'/dev/rbd/rbd/rext4'), dbus.String(u'rext4'), dbus.Int32(-1), dbus.Int32(-1))
      2017-03-23 16:11:26,965 - INFO - openattic_systemd#wrapper - [27005/:1.13534] Deferring call to /volumes::e4fs_format(dbus.String(u'/dev/rbd/rbd/rext4'), dbus.String(u'rext4'), dbus.Int32(-1), dbus.Int32(-1))
      2017-03-23 16:11:26,966 - INFO - openattic_systemd#loggedfunc - Calling /volumes::write_fstab(dbus.Boolean(False), dbus.Int32(0))
      2017-03-23 16:11:26,966 - INFO - openattic_systemd#wrapper - [27005/:1.13534] Deferring call to /volumes::write_fstab(dbus.Boolean(False), dbus.Int32(0))
      2017-03-23 16:11:26,968 - INFO - openattic_systemd#loggedfunc - Calling /volumes::fs_mount(dbus.String(u'ext4'), dbus.String(u'/dev/rbd/rbd/rext4'), dbus.String(u'/media/rext4'), dbus.Array([], signature=dbus.Signature('as')))
      2017-03-23 16:11:26,968 - INFO - openattic_systemd#wrapper - [27005/:1.13534] Deferring call to /volumes::fs_mount(dbus.String(u'ext4'), dbus.String(u'/dev/rbd/rbd/rext4'), dbus.String(u'/media/rext4'), dbus.Array([], signature=dbus.Signature('as')))
      2017-03-23 16:11:26,970 - INFO - openattic_systemd#loggedfunc - Calling /volumes::fs_chown(dbus.String(u'/media/rext4'), dbus.String(u'openattic'), dbus.String(u'users'))
      2017-03-23 16:11:26,970 - INFO - openattic_systemd#wrapper - [27005/:1.13534] Deferring call to /volumes::fs_chown(dbus.String(u'/media/rext4'), dbus.String(u'openattic'), dbus.String(u'users'))
      2017-03-23 16:11:26,971 - INFO - openattic_systemd#loggedfunc - Calling run_queue_background()
      2017-03-23 16:11:26,975 - INFO - openattic_systemd#_run_queue - [5076/:1.13534] Incoming Queue Dump:
      2017-03-23 16:11:26,976 - INFO - openattic_systemd#_run_queue - [5076/:1.13534] -> /nagios::restart_service()
      2017-03-23 16:11:26,976 - INFO - openattic_systemd#_run_queue - [5076/:1.13534] -> /ceph::rbd_create(dbus.String(u'ceph'), dbus.String(u'rbd'), dbus.String(u'rext4'), dbus.Int32(100))
      2017-03-23 16:11:26,976 - INFO - openattic_systemd#_run_queue - [5076/:1.13534] -> /ceph::rbd_map(dbus.String(u'ceph'), dbus.String(u'rbd'), dbus.String(u'rext4'))
      2017-03-23 16:11:26,977 - INFO - openattic_systemd#_run_queue - [5076/:1.13534] -> /nagios::writeconf()
      2017-03-23 16:11:26,977 - INFO - openattic_systemd#_run_queue - [5076/:1.13534] -> /nagios::restart_service()
      2017-03-23 16:11:26,977 - INFO - openattic_systemd#_run_queue - [5076/:1.13534] -> /volumes::e4fs_format(dbus.String(u'/dev/rbd/rbd/rext4'), dbus.String(u'rext4'), dbus.Int32(-1), dbus.Int32(-1))
      2017-03-23 16:11:26,977 - INFO - openattic_systemd#_run_queue - [5076/:1.13534] -> /volumes::write_fstab(dbus.Boolean(False), dbus.Int32(0))
      2017-03-23 16:11:26,977 - INFO - openattic_systemd#_run_queue - [5076/:1.13534] -> /volumes::fs_mount(dbus.String(u'ext4'), dbus.String(u'/dev/rbd/rbd/rext4'), dbus.String(u'/media/rext4'), dbus.Array([], signature=dbus.Signature('as')))
      2017-03-23 16:11:26,977 - INFO - openattic_systemd#_run_queue - [5076/:1.13534] -> /volumes::fs_chown(dbus.String(u'/media/rext4'), dbus.String(u'openattic'), dbus.String(u'users'))
      2017-03-23 16:11:26,978 - INFO - openattic_systemd#_run_queue - [5076/:1.13534] End of queue dump.
      2017-03-23 16:11:26,978 - INFO - openattic_systemd#_run_queue - [5076/:1.13534] Acquiring lock '/var/lock/openattic/volume-142baf21-8f24-414b-9aa4-b4a2bd56c2f5'...
      2017-03-23 16:11:26,979 - INFO - openattic_systemd#_run_queue - [5076/:1.13534] Acquired lock '/var/lock/openattic/volume-142baf21-8f24-414b-9aa4-b4a2bd56c2f5'.
      2017-03-23 16:11:26,979 - INFO - openattic_systemd#_run_queue - [5076/:1.13534] Executing deferred call to /nagios::restart_service()
      2017-03-23 16:11:26,982 - INFO - openattic_systemd#loggedfunc - Calling /volumes::fs_stat(dbus.String(u'/media/rext4'))
      2017-03-23 16:11:26,984 - ERROR - root#acquire_lock - /var/lock/openattic/volume-142baf21-8f24-414b-9aa4-b4a2bd56c2f5 has been locked for more than 0 seconds (PID 5076)
      2017-03-23 16:11:27,007 - INFO - openattic_systemd#service_command - service_command(nagios3): invoking `service nagios3 reload`
      2017-03-23 16:11:27,063 - INFO - openattic_systemd#_run_queue - [5076/:1.13534] Executing deferred call to /ceph::rbd_create(dbus.String(u'ceph'), dbus.String(u'rbd'), dbus.String(u'rext4'), dbus.Int32(100))
      2017-03-23 16:11:27,113 - INFO - openattic_systemd#_run_queue - [5076/:1.13534] Executing deferred call to /ceph::rbd_map(dbus.String(u'ceph'), dbus.String(u'rbd'), dbus.String(u'rext4'))
      2017-03-23 16:11:27,154 - INFO - openattic_systemd#loggedfunc - Calling /volumes::fs_stat(dbus.String(u'/media/eeext4'))
      2017-03-23 16:11:27,200 - INFO - openattic_systemd#loggedfunc - Calling /volumes::fs_stat(dbus.String(u'/media/eext4'))
      2017-03-23 16:11:27,243 - INFO - openattic_systemd#loggedfunc - Calling /volumes::fs_stat(dbus.String(u'/media/exfs'))
      2017-03-23 16:11:27,269 - INFO - openattic_systemd#loggedfunc - Calling /volumes::fs_stat(dbus.String(u'/media/rext4'))
      2017-03-23 16:11:27,270 - ERROR - root#acquire_lock - /var/lock/openattic/volume-142baf21-8f24-414b-9aa4-b4a2bd56c2f5 has been locked for more than 0 seconds (PID 5076)
      2017-03-23 16:11:27,292 - ERROR - root#acquire_lock - /var/lock/openattic/volume-df840a51-80e7-4aaa-9a26-073d1050a0ad has been locked for more than 0 seconds (PID 4832)
      2017-03-23 16:11:27,313 - INFO - openattic_systemd#loggedfunc - Calling /volumes::fs_stat(dbus.String(u'/media/tank'))
      2017-03-23 16:11:49,247 - ERROR - openattic_systemd#invoke - > "rbd" "-c" "/etc/ceph/ceph.conf" "-p" "rbd" "map" "rlun"
      E rbd: sysfs write failed
      E rbd: map failed: (5) Input/output error
      O In some cases useful info is found in syslog - try "dmesg | tail" or so.
      2017-03-23 16:11:49,247 - ERROR - openattic_systemd#_run_queue - Received error:
      Traceback (most recent call last):
      File "/srv/openattic/backend/systemd/management/commands/runsystemd.py", line 117, in _run_queue
      func(scope, *args, **kwargs)
      File "/srv/openattic/backend/ceph/systemapi.py", line 118, in rbd_map
      invoke(["rbd", "-c", "/etc/ceph/%s.conf" % cluster, "-p", pool, "map", image])
      File "/srv/openattic/backend/systemd/procutils.py", line 116, in invoke
      raise SystemError("%s failed: %s" % (cmdline, procerr))
      SystemError: "rbd" "-c" "/etc/ceph/ceph.conf" "-p" "rbd" "map" "rlun" failed: rbd: sysfs write failed
      rbd: map failed: (5) Input/output error
      
      
      2017-03-23 16:11:49,248 - INFO - openattic_systemd#_release_acquired_locks - [4832/:1.13562] Releasing acquired lock '/var/lock/openattic/volume-df840a51-80e7-4aaa-9a26-073d1050a0ad'.
      2017-03-23 16:11:52,323 - INFO - taskqueue.models#transition - Task Transition: Not Started -> Running
      2017-03-23 16:11:52,325 - INFO - taskqueue.models#run_once - Running 4: ceph.tasks.get_rbd_performance_data with [u'38225eb9-bc39-390e-8cae-367ec45af100', u'rbd', u'demo'], {}. Estimated: None
      2017-03-23 16:11:52,405 - INFO - taskqueue.models#finish_task - Task finished: ({u'used_size': 0, u'name': u'demo', u'provisioned_size': 1073741824}, 77.05)
      2017-03-23 16:11:52,405 - INFO - taskqueue.models#transition - Task Transition: Running -> Finished
      2017-03-23 16:11:52,407 - INFO - taskqueue.models#transition - Task Transition: Not Started -> Running
      2017-03-23 16:11:52,409 - INFO - taskqueue.models#run_once - Running 5: ceph.tasks.get_rbd_performance_data with [u'38225eb9-bc39-390e-8cae-367ec45af100', u'rbd', u'rext4'], {}. Estimated: None
      2017-03-23 16:11:52,450 - INFO - taskqueue.models#finish_task - Task finished: ({u'used_size': 0, u'name': u'rext4', u'provisioned_size': 104857600}, 39.45)
      2017-03-23 16:11:52,451 - INFO - taskqueue.models#transition - Task Transition: Running -> Finished
      2017-03-23 16:11:52,453 - INFO - taskqueue.models#transition - Task Transition: Not Started -> Running
      2017-03-23 16:11:52,455 - INFO - taskqueue.models#run_once - Running 6: ceph.tasks.get_rbd_performance_data with [u'38225eb9-bc39-390e-8cae-367ec45af100', u'rbd', u'rlun'], {}. Estimated: None
      2017-03-23 16:11:52,496 - INFO - taskqueue.models#finish_task - Task finished: ({u'used_size': 0, u'name': u'rlun', u'provisioned_size': 104857600}, 38.85)
      2017-03-23 16:11:52,497 - INFO - taskqueue.models#transition - Task Transition: Running -> Finished
      2017-03-23 16:12:41,121 - INFO - openattic_systemd#loggedfunc - Calling ping()
      2017-03-23 16:12:43,555 - INFO - openattic_systemd#loggedfunc - Calling /volumes::fs_stat(dbus.String(u'/media/tank/.snapshots'))
      2017-03-23 16:12:46,547 - INFO - openattic_systemd#loggedfunc - Calling /volumes::fs_stat(dbus.String(u'/media/tank'))
      2017-03-23 16:12:49,238 - ERROR - openattic_systemd#invoke - > "rbd" "-c" "/etc/ceph/ceph.conf" "-p" "rbd" "map" "rext4"
      E rbd: sysfs write failed
      E rbd: map failed: (5) Input/output error
      O In some cases useful info is found in syslog - try "dmesg | tail" or so.
      2017-03-23 16:12:49,239 - ERROR - openattic_systemd#_run_queue - Received error:
      Traceback (most recent call last):
      File "/srv/openattic/backend/systemd/management/commands/runsystemd.py", line 117, in _run_queue
      func(scope, *args, **kwargs)
      File "/srv/openattic/backend/ceph/systemapi.py", line 118, in rbd_map
      invoke(["rbd", "-c", "/etc/ceph/%s.conf" % cluster, "-p", pool, "map", image])
      File "/srv/openattic/backend/systemd/procutils.py", line 116, in invoke
      raise SystemError("%s failed: %s" % (cmdline, procerr))
      SystemError: "rbd" "-c" "/etc/ceph/ceph.conf" "-p" "rbd" "map" "rext4" failed: rbd: sysfs write failed
      rbd: map failed: (5) Input/output error
      

       

      If I follow the path on the system - there is nothing mounted.

      Also the UI provides me with this detail message:

       

      Everything seems to be in order
      The volume is inaccessible.
      The volume is accessible.

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              Unassigned
              Reporter:
              sgmueller Stephan Müller
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: