OpenStack在线调整虚拟机的大小报错
报错日志
2021-06-02 10:15:19.142 844211 INFO nova.compute.manager [req-13a6e58c-9826-49fc-81cb-0221a4febe18 083a5c802fc748658db1821f3f778ddf dd3bc80d13fb431bacc466cbaaf7dbd7 - - -] [instance: ceed63dc-8cdd-43f5-95d8-3f7de86051e9] Setting instance back to ACTIVE after: Instance rollback performed due to: Resize error: not able to execute ssh command: Unexpected error while running command. Command: ssh -o BatchMode=yes 10.89.30.32 mkdir -p /var/lib/nova/instances/ceed63dc-8cdd-43f5-95d8-3f7de86051e9 Exit code: 255 Stdout: u'' Stderr: u'Host key verification failed.\r\n' 2021-06-02 10:15:19.551 844211 INFO nova.compute.manager [req-13a6e58c-9826-49fc-81cb-0221a4febe18 083a5c802fc748658db1821f3f778ddf dd3bc80d13fb431bacc466cbaaf7dbd7 - - -] [instance: ceed63dc-8cdd-43f5-95d8-3f7de86051e9] Successfully reverted task state from None on failure for instance. 2021-06-02 10:15:19.555 844211 ERROR oslo_messaging.rpc.dispatcher [req-13a6e58c-9826-49fc-81cb-0221a4febe18 083a5c802fc748658db1821f3f778ddf dd3bc80d13fb431bacc466cbaaf7dbd7 - - -] Exception during message handling: Resize error: not able to execute ssh command: Unexpected error while running command. Command: ssh -o BatchMode=yes 10.89.30.32 mkdir -p /var/lib/nova/instances/ceed63dc-8cdd-43f5-95d8-3f7de86051e9 Exit code: 255 Stdout: u'' Stderr: u'Host key verification failed.\r\n' 2021-06-02 10:15:19.555 844211 ERROR oslo_messaging.rpc.dispatcher Traceback (most recent call last): 2021-06-02 10:15:19.555 844211 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 138, in _dispatch_and_reply 2021-06-02 10:15:19.555 844211 ERROR oslo_messaging.rpc.dispatcher incoming.message)) 2021-06-02 10:15:19.555 844211 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 185, in _dispatch 2021-06-02 10:15:19.555 844211 ERROR oslo_messaging.rpc.dispatcher return self._do_dispatch(endpoint, method, ctxt, args) 2021-06-02 10:15:19.555 844211 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 127, in _do_dispatch 2021-06-02 10:15:19.555 844211 ERROR oslo_messaging.rpc.dispatcher result = func(ctxt, **new_args) 2021-06-02 10:15:19.555 844211 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/nova/exception.py", line 110, in wrapped 2021-06-02 10:15:19.555 844211 ERROR oslo_messaging.rpc.dispatcher payload) 2021-06-02 10:15:19.555 844211 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ 2021-06-02 10:15:19.555 844211 ERROR oslo_messaging.rpc.dispatcher self.force_reraise() 2021-06-02 10:15:19.555 844211 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise 2021-06-02 10:15:19.555 844211 ERROR oslo_messaging.rpc.dispatcher six.reraise(self.type_, self.value, self.tb) 2021-06-02 10:15:19.555 844211 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/nova/exception.py", line 89, in wrapped 2021-06-02 10:15:19.555 844211 ERROR oslo_messaging.rpc.dispatcher return f(self, context, *args, **kw) 2021-06-02 10:15:19.555 844211 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 359, in decorated_function 2021-06-02 10:15:19.555 844211 ERROR oslo_messaging.rpc.dispatcher LOG.warning(msg, e, instance=instance) 2021-06-02 10:15:19.555 844211 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ 2021-06-02 10:15:19.555 844211 ERROR oslo_messaging.rpc.dispatcher self.force_reraise() 2021-06-02 10:15:19.555 844211 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise 2021-06-02 10:15:19.555 844211 ERROR oslo_messaging.rpc.dispatcher six.reraise(self.type_, self.value, self.tb) 2021-06-02 10:15:19.555 844211 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 328, in decorated_function 2021-06-02 10:15:19.555 844211 ERROR oslo_messaging.rpc.dispatcher return function(self, context, *args, **kwargs) 2021-06-02 10:15:19.555 844211 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 409, in decorated_function 2021-06-02 10:15:19.555 844211 ERROR oslo_messaging.rpc.dispatcher return function(self, context, *args, **kwargs) 2021-06-02 10:15:19.555 844211 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 316, in decorated_function 2021-06-02 10:15:19.555 844211 ERROR oslo_messaging.rpc.dispatcher migration.instance_uuid, exc_info=True) 2021-06-02 10:15:19.555 844211 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ 2021-06-02 10:15:19.555 844211 ERROR oslo_messaging.rpc.dispatcher self.force_reraise() 2021-06-02 10:15:19.555 844211 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise 2021-06-02 10:15:19.555 844211 ERROR oslo_messaging.rpc.dispatcher six.reraise(self.type_, self.value, self.tb) 2021-06-02 10:15:19.555 844211 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 293, in decorated_function 2021-06-02 10:15:19.555 844211 ERROR oslo_messaging.rpc.dispatcher return function(self, context, *args, **kwargs) 2021-06-02 10:15:19.555 844211 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 387, in decorated_function 2021-06-02 10:15:19.555 844211 ERROR oslo_messaging.rpc.dispatcher kwargs['instance'], e, sys.exc_info()) 2021-06-02 10:15:19.555 844211 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ 2021-06-02 10:15:19.555 844211 ERROR oslo_messaging.rpc.dispatcher self.force_reraise() 2021-06-02 10:15:19.555 844211 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise 2021-06-02 10:15:19.555 844211 ERROR oslo_messaging.rpc.dispatcher six.reraise(self.type_, self.value, self.tb) 2021-06-02 10:15:19.555 844211 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 375, in decorated_function 2021-06-02 10:15:19.555 844211 ERROR oslo_messaging.rpc.dispatcher return function(self, context, *args, **kwargs) 2021-06-02 10:15:19.555 844211 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 3933, in resize_instance 2021-06-02 10:15:19.555 844211 ERROR oslo_messaging.rpc.dispatcher self.instance_events.clear_events_for_instance(instance) 2021-06-02 10:15:19.555 844211 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib64/python2.7/contextlib.py", line 35, in __exit__ 2021-06-02 10:15:19.555 844211 ERROR oslo_messaging.rpc.dispatcher self.gen.throw(type, value, traceback) 2021-06-02 10:15:19.555 844211 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 6643, in _error_out_instance_on_exception 2021-06-02 10:15:19.555 844211 ERROR oslo_messaging.rpc.dispatcher raise error.inner_exception 2021-06-02 10:15:19.555 844211 ERROR oslo_messaging.rpc.dispatcher ResizeError: Resize error: not able to execute ssh command: Unexpected error while running command. 2021-06-02 10:15:19.555 844211 ERROR oslo_messaging.rpc.dispatcher Command: ssh -o BatchMode=yes 10.89.30.32 mkdir -p /var/lib/nova/instances/ceed63dc-8cdd-43f5-95d8-3f7de86051e9 2021-06-02 10:15:19.555 844211 ERROR oslo_messaging.rpc.dispatcher Exit code: 255 2021-06-02 10:15:19.555 844211 ERROR oslo_messaging.rpc.dispatcher Stdout: u'' 2021-06-02 10:15:19.555 844211 ERROR oslo_messaging.rpc.dispatcher Stderr: u'Host key verification failed.\r\n' 2021-06-02 10:15:19.555 844211 ERROR oslo_messaging.rpc.dispatcher 2021-06-02 10:15:46.889 844211 INFO nova.compute.resource_tracker [req-2adbb715-5cef-4e83-ac9f-41a39c554fa9 - - - - -] Auditing locally available compute resources for node st-computer12
在日志中错误提示还是挺明显的,根据错误提示,解决相应的故障就行了。
注意一点:
openstack的虚拟机在线调整大小的原理:
其实就相当于做了一个云主机在不同宿主机(计算节点)之间的迁移,所以前提是至少需要有两个计算节点。
如果是单机部署的openstack(即控制节点和计算节点都在一台机器上),有且只有一个计算节点,那么是无法完成在线调整虚拟机大小的。
同时要注意的是:
要在相关迁移云主机间进行无密码访问,由于OpenStack是由Nova组件来管理云主机,所以需要对Nova用户进行无密码访问。
如上图中的centos-004这台云主机的规格采用的是名为kvm001类型,即2核CPU、4G内容,现在需要将这台云主机的规格降为1核2G,即kvm002类型。操作流程如下:
1)修改控制节点和节点节点的nova.conf文件,添加下面两行内容:
# 可以参考最原始的nova.conf [defaults] allow_resize_to_same_host=True scheduler_default_filters=RetryFilter,AvailabilityZoneFilter,RamFilter,ComputeFilter,ComputeCapabilitiesFilter,ImagePropertiesFilter,ServerGroupAntiAffinityFilter,ServerGroupAffinityFilter
2)控制节点上重启nova相关服务
[root@linux-node1 src]# systemctl restart openstack-nova-api.service openstack-nova-cert.service openstack-nova-consoleauth.service openstack-nova-scheduler.service openstack-nova-conductor.service openstack-nova-novncproxy.service
3)计算节点上重启nova服务
[root@linux-node1 nova]# systemctl restart openstack-nova-compute
4)nova账号的ssh双向认证关系
下面的操作在云主机所在计算节点和其他相关迁移云主机的计算节点上操作。
要清楚的是:
计算节点可能有多台,但是我们只需要在要调整大小的云主机所在的计算节点和其他的另外一台或多台计算几点上操作就行,并不是要求所有的计算节点都要操作(全部计算节点都操作也是可以的)
将nova账号的shell类型改为/bin/bash,即可以使用nova账号登陆
[root@linux-node1 ~]# cat /etc/passwd|grep nova nova:x:162:162:OpenStack Nova Daemons:/var/lib/nova:/bin/bash
然后在nova账号下产生公私钥
[root@linux-node2 ~]# su - nova Last login: Thu Nov 17 17:32:54 CST 2016 from linux-node2.openstack on pts/11 -bash-4.2$ ssh-keygen -t rsa -bash-4.2$ pwd /var/lib/nova -bash-4.2$ cd .ssh/ -bash-4.2$ ls id_rsa id_rsa.pub
将id_rsa.pub拷贝为authorized_keys
-bash-4.2$ cp id_rsa.pub authorized_keys ``` 然后将id_rsa.pub公钥内容拷贝到其他相关迁移云主机的计算节点上的/var/lib/nova/.ssh/authorized_keys文件内,同时也要讲对方的公钥内容拷贝过来,做成双向信任关系(即双方在nova账号下ssh登陆时都不需要输入密码) 同时,还要在各计算节点上做nova和root账号的双向信任关系。 修改权限 ```bash [root@linux-node1 ~]# cd /var/lib/nova/.ssh/ [root@linux-node1 .ssh]# ll total 16 -rw-------. 1 nova nova 816 Nov 17 17:32 authorized_keys -rw-------. 1 nova nova 1679 Nov 17 17:12 id_rsa -rw-r--r--. 1 nova nova 408 Nov 17 17:12 id_rsa.pub
修改sudo权限
[root@linux-node1 ~]# visudo ..... nova ALL=(ALL) NOPASSWD: ALL
最后测试nova账号间的信任连接,确认ssh无密码连接!
5)接下来登陆dashboard界面,在线调整云主机的大小:
选择目标flavor
点击确认修改尺寸
待调整后,再查询虚拟机新的大小规格,就会发现已经更新了。