btrfs: qgroup: fix qgroup prealloc rsv leak in subvolume operations

In the Linux kernel, the following vulnerability has been resolved:

btrfs: qgroup: fix qgroup prealloc rsv leak in subvolume operations

Create subvolume, create snapshot and delete subvolume all use

btrfs_subvolume_reserve_metadata() to reserve metadata for the changes

done to the parent subvolume's fs tree, which cannot be mediated in the

normal way via start_transaction. When quota groups (squota or qgroups)

are enabled, this reserves qgroup metadata of type PREALLOC. Once the

operation is associated to a transaction, we convert PREALLOC to

PERTRANS, which gets cleared in bulk at the end of the transaction.

However, the error paths of these three operations were not implementing

this lifecycle correctly. They unconditionally converted the PREALLOC to

PERTRANS in a generic cleanup step regardless of errors or whether the

operation was fully associated to a transaction or not. This resulted in

error paths occasionally converting this rsv to PERTRANS without calling

record_root_in_trans successfully, which meant that unless that root got

recorded in the transaction by some other thread, the end of the

transaction would not free that root's PERTRANS, leaking it. Ultimately,

this resulted in hitting a WARN in CONFIG_BTRFS_DEBUG builds at unmount

for the leaked reservation.

The fix is to ensure that every qgroup PREALLOC reservation observes the

following properties:

1. any failure before record_root_in_trans is called successfully

results in freeing the PREALLOC reservation.

2. after record_root_in_trans, we convert to PERTRANS, and now the

transaction owns freeing the reservation.

This patch enforces those properties on the three operations. Without

it, generic/269 with squotas enabled at mkfs time would fail in ~5-10

runs on my system. With this patch, it ran successfully 1000 times in a

row.