(CentOS 7, Ceph Incunabulus, everything patched up to date)BackgroundI've got a working converged Ceph/qemu/kvm cluster with cephx enabled for authentication. I'm trying to use this to store block devices for qemu/kvm virtual machines - not an uncommon configuration. I'd like to set up a pool for virtual disks in ceph, define that pool and authenticate to it in libvirt, and create virtual machines using the pool. All the tutorials and examples seem to show setting up cephx authentication directly on each disk file. Given my day so far, I'm beginning to see why!QuestionWhen I try the below, qemu-kvm appears to get passed a drive configured as file=rbd:vmlive/testguest-vda:auth_supported=none. That should read auth_supported=cephx, and for some reason it ain't happening!Can I put all the authentication in the pool as I want, or do I have to put an authentication section in each virtual disk's XML? If I can do this, where's my mistake in what's below?What I'm trying nowContextI have two nodes: test1 and test2. Each has two OSDs, each is also a monitor (yes, I'm aware running with an even number of monitors is a bad idea). Each has libvirt installed, hence qemu/kvm. I'm running the commands below as root on test1. The UUID has been changed, and you'll see I don't make any keys visible below!Create a Ceph pool and a client.kvm userSENSIBLE_PGS=512 # Calculate as OSDs * 100 / number of replicas, rounded up to next power of 2ID=kvmPOOL=vmliveceph osd pool create $POOL $SENSIBLE_PGS $SENSIBLE_PGSKEY=`ceph auth get-or-create client.$ID | grep -v '^\[' | awk '{print $3}'`ceph auth caps client.$ID mon allow r osd allow rwx pool=$POOLCreate a libvirt secret holding the Ceph user's keySECRET_DEFINITION_FILE=~/secret-$ID.xmlSECRET_UUID=12345678-1234-1234-1234-123456789abccat > $SECRET_DEFINITION_FILE << EOF<secret ephemeral='no' private='no'>  <uuid>$SECRET_UUID</uuid>  <usage type='ceph'>    <name>ceph client.$ID key</name>  </usage></secret>EOFvirsh secret-define --file $SECRET_DEFINITION_FILErm -f $SECRET_DEFINITION_FILEvirsh secret-set-value $SECRET_UUID $KEYCreate a libvirt storage pool, trying to use cephx authenticationPOOL_DEFINITION_FILE=~/pool-$POOL.xmlcat > $POOL_DEFINITION_FILE << EOF<pool type=rbd>  <name>$POOL</name>  <source>    <name>$POOL</name>    <host name=test1 port=6789 />    <host name=test2 port=6789 />    <auth username='$ID' type='ceph'>      <secret uuid='$SECRET_UUID'/>    </auth>  </source></pool>EOFvirsh pool-define $POOL_DEFINITION_FILErm -f $POOL_DEFINITION_FILEvirsh pool-autostart $POOLvirsh pool-start $POOLCreate a guest using virt-installNAME=testguestDEV=vdaSIZE=8GIMAGE=CentOS-7-x86_64-Minimal-1503-01.isoVCPUS=1RAM=512MACLAST_HEX=10IPLAST_DECIMAL=16FILE=$NAME-$DEVqemu-img create -f rbd rbd:$POOL/$FILE $SIZEsudo virsh net-update default add-last ip-dhcp-host \  --xml <host mac='52:54:00:00:00:$MACLAST_HEX' ip='192.168.122.$IPLAST_DECIMAL' name='$NAME'/> --live --configsudo virt-install \  --connect qemu:///system \  --virt-type kvm \  --name $NAME \  --ram $RAM \  --vcpus=$VCPUS \  --disk vol=$POOL/$FILE \  --location /var/lib/libvirt/images/$IMAGE \  --vnc \  --noautoconsole \  --os-type linux \  --os-variant rhel7 \  --network=bridge:virbr0,model=virtio,mac=52:54:00:00:00:$MACLAST_HEX \  --autostartThe resultWARNING  vmlive/testguest-vda may not be accessible by the hypervisor. You will need to grant the 'qemu' user search permissions for the following directories: ['vmlive', '']Starting install...Retrieving file .treeinfo...Retrieving file vmlinuz...Retrieving file initrd.img...ERROR    internal error: process exited while connecting to monitor: 2016-01-06T15:16:54.639890Z qemu-kvm: -drive file=rbd:vmlive/testguest-vda:auth_supported=none:mon_host=test1\:6789,if=none,id=drive-virtio-disk0,format=raw: error connecting2016-01-06T15:16:54.640574Z qemu-kvm: -drive file=rbd:vmlive/testguest-vda:auth_supported=none:mon_host=test1\:6789,if=none,id=drive-virtio-disk0,format=raw: could not open disk image rbd:vmlive/testguest-vda:auth_supported=none:mon_host=test1\:6789: Could not open 'rbd:vmlive/testguest-vda:auth_supported=none:mon_host=test1\:6789': Operation not supportedDomain installation does not appear to have been successful.If you've read this far... thanks, congratulations, and all help gratefully received!Update 1Update: This appears to be a problem with virt-install partially (but incompletely) parsing pools.  If I use --print-xml the partial XML generated for the disk is:<disk type=network device=disk>  <driver name=qemu/>  <source protocol=rbd name=vmlive/testguest-vda>    <host name=test1 port=6789/>  </source>  <target dev=vda bus=virtio/></disk>... which includes much of the pool definition, but not the auth stanza.I'll keep investigating.
Can I enable cephx authentication using a pool in qemu/kvm?
centos;kvm;qemu;ceph
This is presently a known bug in virt-manager - see the email thread at https://www.redhat.com/archives/virt-tools-list/2016-January/msg00007.html.In the meantime, I'll find another way round the problem.