- 积分
- 16840
在线时间 小时
最后登录1970-1-1
|
马上注册,结交更多好友,享用更多功能,让你轻松玩转社区。
您需要 登录 才可以下载或查看,没有账号?开始注册
x
关于openstack中虚拟机VNC访问安全问题2 l2 p2 u( ]' x4 {! z& l
前几天收到甲方给出的文件信息,线上OpenStack环境的物理机VNC端口能够自由访问,要求整改。
& n- U$ |: E5 D( y! R& u9 v" H随机抽查了两台宿主机的VNC端口,确实很多业务的同学使用noVNC后没有退出终端的习惯,往往都是用完了就直接关闭窗口。不得不说这样隐患很大啊,首先不说通过外部方式规避风险,如果内网里面有一些script kiddie随时都能将我们线上的虚拟机VNC端口扫出来干些坏事。我这里也用过nmap测试了下开发环境的网络端口,如下:6 Z [2 V; M: @$ A2 T
[root@controller1 ~]# nmap 10.161.53.1/ E) X/ B) W9 l2 U7 d8 a# \( u% T
Starting Nmap 6.40 ( http://nmap.org ) at 2022-01-17 09:46 CST
' a0 E5 N2 H- i7 z& FNmap scan report for compute10 (10.161.53.1)
, P( R! n1 X1 OHost is up (0.000080s latency).) N# }$ P+ v! Q# X p5 C X
Not shown: 989 closed ports: g7 s H+ G+ A; I! |1 d
PORT STATE SERVICE* g, e$ C! U% e( r" U8 P; [) ]9 `
22/tcp open ssh
8 I4 _0 X% R; d" @3 q( a5900/tcp open vnc
/ I3 @, M2 n% j2 ^5901/tcp open vnc-1
7 w- k u7 I# k; l- B0 e$ V$ R/ j9 j5902/tcp open vnc-2( v1 r* v2 o, l4 ?# d" o+ o8 R
5903/tcp open vnc-3
' S, J- C* n% \5 d1 _' o O2 f/ c5904/tcp open unknown
) z& D' U. t+ i5 y9 I5 C, S5906/tcp open unknown* j$ E4 P% S& o" l/ l9 p
5907/tcp open unknown U9 r6 ]) f9 l; f) K
5910/tcp open cm
9 r7 D+ G8 F) V3 i& J6 }: e6 c8 {5911/tcp open cpdlc ]% \% J6 ~7 @! ^( B% u
8022/tcp open oa-system
+ G: W# Z$ K# g8 KMAC Address: D4:5D:64:08:45:02 (Unknown)
( E; M7 f2 x" ?) [, f3 y' ENmap done: 1 IP address (1 host up) scanned in 1.65 seconds# M* U0 F( h9 I( y8 b; [9 z
[root@controller1 ~]# nmap 10.161.53.2, @+ a0 T/ U9 p. Y( B
Starting Nmap 6.40 ( http://nmap.org ) at 2022-01-17 09:46 CST9 {2 B1 x( u, K& |4 }
Nmap scan report for compute11 (10.161.53.2) o4 p; b+ n& {/ i2 k
Host is up (0.000070s latency).5 R( a+ t: i- B
Not shown: 989 closed ports- f* x" u% R+ h2 Z3 K; e
PORT STATE SERVICE% B$ H5 G5 D5 }4 C3 f
22/tcp open ssh
6 l5 |+ Z& B0 r* }5900/tcp open vnc
7 o! V4 A* t+ z" h( B; U9 D5901/tcp open vnc-1- M8 h: @2 e% H0 w& U
5902/tcp open vnc-2) l9 F" M$ d5 I3 P# J
5903/tcp open vnc-3
/ ^7 X0 n$ T+ F: `' i/ ?, u' \5 P& y5904/tcp open unknown
6 q! W$ P- S7 J0 G* }5906/tcp open unknown
$ r) W1 M* W6 B; q5907/tcp open unknown
$ |/ g9 i! t {/ S$ b5911/tcp open cpdlc
) F# P! s* {( a: [5915/tcp open unknown
2 d. t0 E7 L4 j8022/tcp open oa-system
5 O8 N. B* b4 z4 d, wMAC Address: D4:5D:64:07:B3:DA (Unknown)
, a5 D& s" y3 ~. z' ]Nmap done: 1 IP address (1 host up) scanned in 1.67 seconds5 ] f, K) z; ~+ S; J" o
[root@controller1 ~]# nmap 10.161.53.3
, E& ?( r; S4 P( V, \! ZStarting Nmap 6.40 ( http://nmap.org ) at 2022-01-17 09:46 CST
, N! p# F: }" f' q) q6 ~Nmap scan report for compute01 (10.161.53.3)
7 d: Z( k( i' R& KHost is up (0.000076s latency).
- L7 c$ C& X" Y1 INot shown: 992 closed ports: D; c/ \7 h$ ^% d- M& t8 K
PORT STATE SERVICE. o. l; \/ y9 }% {4 t5 t7 h( v
22/tcp open ssh# {! N% `" r" }& t6 w( o' I3 U4 c
5900/tcp open vnc/ Y* A W1 L& v% _7 q9 S5 A: R
5901/tcp open vnc-16 Z2 k7 z' ]2 l2 @% l' F9 e6 i# z/ s; _
5902/tcp open vnc-2
3 Y9 W1 I' u$ L4 v& A5903/tcp open vnc-3
2 ]# o" }1 S+ L- e& ?5904/tcp open unknown
3 L2 A7 q0 \( ]- ?# c" W: c5 d- Z5906/tcp open unknown
2 C& n1 t0 d; @4 S9 x$ W8022/tcp open oa-system% p6 O) h5 r' U# ?. H( R! |" J) a
MAC Address: D4:5D:64:08:45:5E (Unknown)
2 P( G f7 D2 e5 MNmap done: 1 IP address (1 host up) scanned in 1.68 seconds) V2 X: r. N- L" i! D3 o& V
[root@controller1 ~]# nmap 10.161.53.4
- s, U) y, q& d9 BStarting Nmap 6.40 ( http://nmap.org ) at 2022-01-17 09:46 CST
# p+ V" I, M1 |7 {: rNmap scan report for compute02 (10.161.53.4)
- Z( W% h0 j6 h7 RHost is up (0.000081s latency).
' s9 ], v# t- q- A$ F& D1 I6 MNot shown: 988 closed ports' m8 t3 u2 V. t' v0 j' u6 f* G' X
PORT STATE SERVICE; r& q2 e5 W6 ]. I
22/tcp open ssh
* B# l/ k, t, ^5900/tcp open vnc
% J I7 I o6 a- q# w( Z5901/tcp open vnc-1
5 C; H; X1 M: E' {" j5902/tcp open vnc-2
7 l# T% ]* o+ P9 Z5903/tcp open vnc-3# u7 _0 q& N$ U7 A$ a( g/ X/ k
5904/tcp open unknown" W; R3 h1 d2 |# U+ o% \; K3 u
5906/tcp open unknown
& u3 _9 F! Z* s& _2 x5907/tcp open unknown: D/ e9 T$ R P3 a
5910/tcp open cm
6 N$ K# ~& {2 t1 u: K. N6 M5911/tcp open cpdlc
& `- D; H* f% B2 o% ]3 _5915/tcp open unknown
* a1 M- ]) p3 {8022/tcp open oa-system
* L" V' g3 u! K7 }: xMAC Address: D4:5D:64:08:45:D6 (Unknown). L& V) ~ `1 `5 n2 j/ E
Nmap done: 1 IP address (1 host up) scanned in 1.67 seconds0 `; H+ |3 Q8 ]+ d9 ^7 x
[root@controller1 ~]# nmap 10.161.53.5* q1 `# K' O& i
Starting Nmap 6.40 ( http://nmap.org ) at 2022-01-17 09:47 CST
8 t# w# p6 M- t/ @9 P$ r" y1 f" |4 QNmap scan report for compute03 (10.161.53.5)% A" d2 Q( i6 f+ L5 C8 z. {( D# U+ ]
Host is up (0.000082s latency).
6 t. j% \. ~5 w3 r9 U' JNot shown: 993 closed ports+ ~' T& ^- p6 p: t5 }
PORT STATE SERVICE
7 l. L" B* b5 E- \4 C4 G, ~22/tcp open ssh
8 J" X7 I& }8 B5900/tcp open vnc- `2 h1 d3 C* t( t2 c+ P
5901/tcp open vnc-1
4 {3 R7 ?( u/ B/ J5902/tcp open vnc-2
, d8 b/ Y1 s" G5 ^5903/tcp open vnc-3* [5 |3 \) p5 I5 k7 y5 k A
5904/tcp open unknown
% f7 {1 e3 \7 |4 M/ A1 G8022/tcp open oa-system
5 [' N' [% p0 H. [MAC Address: D4:5D:64:08:44:DE (Unknown)
8 i4 J5 H% s. a/ o7 q/ J" VNmap done: 1 IP address (1 host up) scanned in 1.67 seconds
* l0 ^/ {6 a" R; d; V[root@controller1 ~]# nmap 10.161.53.69 F1 J% N6 u8 }, x- _/ N7 p
Starting Nmap 6.40 ( http://nmap.org ) at 2022-01-17 09:47 CST
3 r4 j9 q+ P9 j: dNmap scan report for controller1 (10.161.53.6) J t, M: l. `0 E X; |
Host is up (0.000014s latency)." `3 t5 _1 ^5 o: f
Not shown: 992 closed ports
0 z3 b" C, d) N% d# E6 O2 hPORT STATE SERVICE* t" D0 o3 E: Q5 x* K
22/tcp open ssh" {5 p4 Z; t" t! P) \7 d& b
80/tcp open http
8 |- K; f: ?1 Q8 _* Z0 _ ^7 E8 K U1984/tcp open bigbrother8 z! W( X3 b% u! w* Y
3306/tcp open mysql
+ v) T$ w, q& E) u4000/tcp filtered remoteanything5 H7 m. v& [ z/ Y8 J' H1 P5 M
4567/tcp open tram5 G, D9 z# Z, n3 q1 f
5000/tcp open upnp) \! r% E; f2 Q/ I, T3 d
5001/tcp open commplex-link
5 @7 k, @& c4 ?) B% j- VNmap done: 1 IP address (1 host up) scanned in 2.75 seconds
, W( Z0 C( K( ~: [. ][root@controller1 ~]# nmap 10.161.53.7/ Q1 n+ W4 X% X
Starting Nmap 6.40 ( http://nmap.org ) at 2022-01-17 09:47 CST
" A2 j9 ^' g1 c( l& J" tNmap scan report for controller2 (10.161.53.7)% U8 O. w. v1 s- T0 l
Host is up (0.000078s latency).3 s/ j. b- I* ` G
Not shown: 993 closed ports
1 `7 p$ d6 K6 V9 Y4 j! n9 XPORT STATE SERVICE
9 r; k$ H4 Z4 J4 s! F' t8 S22/tcp open ssh& G% A& m+ |$ s6 O: Q2 F
80/tcp open http) h& y; b8 }8 D ~- h/ s: F6 O, G5 A
1984/tcp open bigbrother
% J8 h2 n# ~2 L: ?. H+ B3306/tcp open mysql
- z6 a! ?% }. n" N3 k: i4567/tcp open tram
* u% y# n2 Z; P0 c# v- @5000/tcp open upnp
8 S4 M: G# r6 _6 i8 U5001/tcp open commplex-link' [3 \2 m2 J/ X# J- N& { D0 w6 @# d
MAC Address: D4:5D:64:08:45:0A (Unknown)- h6 p, i* W1 `5 a# }9 x
Nmap done: 1 IP address (1 host up) scanned in 1.55 seconds* Y; k5 Z& M4 K7 G- G7 F# ~, n' b
F H* B" E% s- P; {4 J
结果太恐怖了吧,如果有业务同学在使用noVNC之后没有退出终端,那么另一个人如果知道了宿主机的IP和端口是完全可以登录这台虚拟机的,直接操作虚机,后果不堪设想:, ~1 }1 L+ G+ t: F2 k/ J. x
[& i- e- k, J5 i5 [2 g1 ~5 `
解决
3 \8 P2 o' b* @1 m0 A知道了问题,那就有对应的解决方案!
4 a. u3 A8 ]8 O4 W, I+ a目前我们暂时只想到两个方法来解决这个问题,其他方法还有待大牛给予指导:
J5 a4 B7 m) a d. v" L o
" x( G( \5 i: Y) `方案一
+ v& o4 ^5 W+ E通过firewalld限制INPUT表对5900:6000的访问规则
! f( ~" b2 P. ^, ]4 ~% C2 j8 [方案二0 w. z$ U3 `* u$ f# |% z
添加密码访问VNC
8 P) R% ~3 o8 b! O
2 t' b P% e- {# ~7 s. }7 D操作
: J- e7 Z4 O. y+ ]5 ^firewalld
; O1 i2 u6 o( m$ N我们知道OpenStack通过VNC Proxy将管理网和业务网隔离开来,以便我们可以使用管理网络的6080端口访问虚拟机VNC,同时提供Token用于验证访问的合法性。一个VNC Proxy在OpenStack里的处理流程如下:
1 L7 i' a* V# m# s& c( k3 X7 x' VVNC Porxy处理流程! D9 B4 i) v9 h
1. 一个用户试图从浏览器里面打开连接到虚拟机的VNC Client
( W* ]) L+ L! @' j9 u! {2. 浏览器向nova-api发送请求,要求返回访问vnc的url3 D- ~7 B3 K$ J8 m
3. nova-api调用nova-compute的get vnc console方法,要求返回连接VNC的信息: `8 r' z0 Y6 L A3 P; E
4.nova-compute调用libvirt的get vnc console函数# Y) J- c4 W0 m8 S/ g, a+ X" {
5.libvirt会通过解析虚拟机运行的/etc/libvirt/qemu/instance-0000000c.xml文件来获得VNC Server的信息7 S" e$ m& J% a R( T8 [
6.libvirt将host, port等信息以json格式返回给nova-compute3 T1 ?% f* C, J2 U0 g; f, V& I
7.nova-compute会随机生成一个UUID作为Token
+ w9 }3 m; s2 H* S* I' H: O1 n8.nova-compute将libvirt返回的信息以及配置文件中的信息综合成connect_info返回给nova-api
) x9 @5 ^7 L9 g. q! @( Q& U9.nova-api会调用nova-consoleauth的authorize_console函数0 t* v# r1 N1 t( N% b
10.nova-consoleauth会将instance –> token, token –> connect_info的信息cache起来9 M* I) e2 a: @7 T, J' w! b7 M
11.nova-api将connect_info中的access url信息返回给浏览器:http://contorller:6080/vnc_auto.html?token=7efaee3f-eada-4731-a87c-e173cbd25e98&title=helloworld%289169fdb2-5b74-46b1-9803-60d2926bd97c%29% {1 ]9 m$ d- v0 D: V
12.浏览器会试图打开这个链接
/ Q+ j" U* u3 {( ~& n13.这个链接会将请求发送给nova-novncproxy
+ @" M4 J: b, I+ A( ~4 O* o14.nova-novncproxy调用nova-consoleauth的check_token函数3 J7 f% T9 y. S H5 V- A9 S
15.nova-consoleauth验证了这个token,将这个instance对应的connect_info返回给nova-novncproxy
( |- }# n7 {) T9 u+ b16.nova-novncproxy通过connect_info中的host, port等信息,连接compute节点上的VNC Server,从而开始了proxy的工作
9 J+ ^% C& n0 Z3 X这里重要的就是第16步, nova-novncproxy是通过连接host:vncport的方式提供vnc访问服务。
- L: d' ]4 Y# K9 W+ Q; N& Y7 q那么也就是说,计算节点的VNC端口只需要让nova-novncporxy服务能够访问就行,有了这个就好办了。1 e; K* s( [0 l" p0 F
操作firewalld& g1 c4 c' z2 M: p# _3 y b7 N
在所有计算节点firewalld的INPUT表中添加如下规则:+ t0 n! U& g5 |; }
1 G7 }1 F9 @2 d' o* x' B检查防火墙是否启动:
% i1 T: ?6 J6 _8 Nsystemctl status firewalld.service# X) ?7 K$ y+ {" f+ ^" c
启动防火墙:$ u3 f2 l+ i3 k. Z
systemctl start firewalld.service
; m1 z/ V/ Y" W+ m8 [7 n
; s, m; J) I0 m+ o
- n0 l+ F7 ?# x! ~开机启动防火墙:: v9 F7 K/ j# m" ^) ? w
systemctl enable firewalld.service
j3 T; R4 w# r# }- N2 T7 j添加规则:2 F' O4 E$ n/ l; h
firewall-cmd --permanent --add-rich-rule="rule family="ipv4" source address="10.161.53.6" port protocol="tcp" port="5900-6000" accept"
) `5 U. T- p2 R+ Z firewall-cmd --permanent --add-rich-rule="rule family="ipv4" source address="10.161.53.100" port protocol="tcp" port="5900-6000" accept"
; G. w1 \1 J5 x firewall-cmd --reload
" T" v( T0 m6 h' v4 y firewall-cmd --list-all
- r F- M0 g+ t7 l删除规则:
1 a- h1 l j" q5 u3 O firewall-cmd --permanent --remove-rich-rule="rule family="ipv4" source address="10.161.53.6" port protocol="tcp" port="5900-6000" accept") K1 R# j; \3 n7 I% G
^1 J4 t) Z+ n D! A; e7 { firewall-cmd --reload ; F- ]6 b4 v# _
firewall-cmd --list-all D7 @/ Q" R- v, c* d
, } @. [5 I0 d; F0 @( H
. q7 R- W7 C5 ?& n( S
添加防火墙规则:
; ~0 x8 a4 Q& `9 k4 Tfirewall-cmd --permanent --add-rich-rule="rule family="ipv4" source address="10.161.53.31/27" port protocol="tcp" port="5900-6000" accept"
( Q7 {4 }: L( w; @
# f% w4 e- `8 J# cfirewall-cmd --reload
. D' \ P. G# q2 x4 H9 r: [) \- j0 y, k
通过测试上面开启的firewalld会导致业务无法访问。5 m& d: A, [; {
这里改变方式使用iptables的规则吧:
* m) `, W' ?/ H* R' O2 }2 M2 Y3 h5 R9 O2 o8 ?4 b- a. U- ~
iptables -A INPUT -s 10.161.53.31/27 -p tcp -m multiport --dports 5900:5999 -m comment --comment "ACCEPT VNC Port only by Controller Node" -j ACCEPT
% a: w6 Q4 V& b3 v% y iptables -L. ~1 e2 U' H3 Q0 N
iptables -A INPUT -p tcp -m multiport --dports 5900:5999 -j REJECT --reject-with icmp-port-unreachable H1 x$ m; b6 S; x9 \: m
% E3 R0 b9 l6 s
" Y3 u' n) L9 n6 ]
+ u" u' l! X) {+ o0 O# y V7 b& W |
|