fbdev: Fix do_register_framebuffer to prevent null-ptr-deref in fb_videomode_to_var

In the Linux kernel, the following vulnerability has been resolved:

fbdev: Fix do_register_framebuffer to prevent null-ptr-deref in fb_videomode_to_var

If fb_add_videomode() in do_register_framebuffer() fails to allocate

memory for fb_videomode, it will later lead to a null-ptr dereference in

fb_videomode_to_var(), as the fb_info is registered while not having the

mode in modelist that is expected to be there, i.e. the one that is

described in fb_info->var.

================================================================

general protection fault, probably for non-canonical address 0xdffffc0000000001: 0000 [#1] PREEMPT SMP KASAN NOPTI

KASAN: null-ptr-deref in range [0x0000000000000008-0x000000000000000f]

CPU: 1 PID: 30371 Comm: syz-executor.1 Not tainted 5.10.226-syzkaller #0

Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.12.0-1 04/01/2014

RIP: 0010:fb_videomode_to_var+0x24/0x610 drivers/video/fbdev/core/modedb.c:901

Call Trace:

display_to_var+0x3a/0x7c0 drivers/video/fbdev/core/fbcon.c:929

fbcon_resize+0x3e2/0x8f0 drivers/video/fbdev/core/fbcon.c:2071

resize_screen drivers/tty/vt/vt.c:1176 [inline]

vc_do_resize+0x53a/0x1170 drivers/tty/vt/vt.c:1263

fbcon_modechanged+0x3ac/0x6e0 drivers/video/fbdev/core/fbcon.c:2720

fbcon_update_vcs+0x43/0x60 drivers/video/fbdev/core/fbcon.c:2776

do_fb_ioctl+0x6d2/0x740 drivers/video/fbdev/core/fbmem.c:1128

fb_ioctl+0xe7/0x150 drivers/video/fbdev/core/fbmem.c:1203

vfs_ioctl fs/ioctl.c:48 [inline]

__do_sys_ioctl fs/ioctl.c:753 [inline]

__se_sys_ioctl fs/ioctl.c:739 [inline]

__x64_sys_ioctl+0x19a/0x210 fs/ioctl.c:739

do_syscall_64+0x33/0x40 arch/x86/entry/common.c:46

entry_SYSCALL_64_after_hwframe+0x67/0xd1

================================================================

Even though fbcon_init() checks beforehand if fb_match_mode() in

var_to_display() fails, it can not prevent the panic because fbcon_init()

does not return error code. Considering this and the comment in the code

about fb_match_mode() returning NULL - "This should not happen" - it is

better to prevent registering the fb_info if its mode was not set

successfully. Also move fb_add_videomode() closer to the beginning of

do_register_framebuffer() to avoid having to do the cleanup on fail.

Found by Linux Verification Center (linuxtesting.org) with Syzkaller.