Uploaded image for project: 'openATTIC'
  1. openATTIC
  2. OP-2057

RBD: ceph.tasks.get_rbd_performance_data Tasks fails if RBD was deleted

    XMLWordPrintable

    Details

      Description

      Looks like the Nagios check for an RBD may not get deleted, if the RBD was removed:

      017-04-06 19:32:35,278 - INFO - taskqueue.models#transition - Task Transition: Not Started -> Running
      2017-04-06 19:32:35,281 - INFO - taskqueue.models#run_once - Running 14: ceph.tasks.get_rbd_performance_data with [u'xxx', u'test', u'rbd'], {}. Estimated: None
      2017-04-06 19:32:35,325 - ERROR - taskqueue.models#run_once - Failed to run "ceph.tasks.get_rbd_performance_data with [u'xxx', u'test', u'rbd'], {}" created "2017-04-06 19:32:31.574484"
      Traceback (most recent call last):
        File "/usr/share/openattic/taskqueue/models.py", line 79, in run_once
          res = task.run_once()
        File "/usr/share/openattic/taskqueue/models.py", line 239, in run_once
          res = self.wrapper.call_now(*self.args, **self.kwargs)
        File "/usr/share/openattic/taskqueue/models.py", line 316, in call_now
          return self._orig_func(*args, **kwargs)
        File "/usr/share/openattic/ceph/tasks.py", line 77, in get_rbd_performance_data
          disk_usage = api.image_disk_usage(pool_name, image_name)
        File "/usr/share/openattic/ceph/librados.py", line 779, in image_disk_usage
          '--pool', pool_name, '--image', name, '--format', 'json'])
        File "/usr/lib64/python2.7/subprocess.py", line 219, in check_output
          raise CalledProcessError(retcode, cmd, output=output)
      CalledProcessError: Command '['rbd', 'disk-usage', '--cluster', 'ceph', '--pool', u'test', '--image', u'rbd', '--format', 'json']' returned non-zero exit status 2
      2017-04-06 19:32:35,329 - INFO - taskqueue.models#finish_task - Task finished: Command '['rbd', 'disk-usage', '--cluster', 'ceph', '--pool', u'test', '--image', u'rbd', '--format', 'json']' returned non-zero exit status 2
      2017-04-06 19:32:35,329 - INFO - taskqueue.models#transition - Task Transition: Running -> Exception
      

      In the meantime this pool and rbd is deleted and does not exist anymore..

      However - we see in the log still errors (op systemd log) against the deleted pool with ceph.tasks.get_rbd_performance_data.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                swagner Sebastian Wagner
                Reporter:
                swagner Sebastian Wagner
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: