Assuming the following scenario:

Customer Sale Order with i.e. two lines, Product A and Product B.

1. Create a regular Invoice for that Sale Order (assuming everything is invoicable) and confirm it
2. Create a Credit Note from that Invoice and confirm it

Odoo will reconcile the Credit Note (return payment) against the Invoice, resulting in a Paid Credit Note and a Partially Paid Invoice.

Now, the Sale Order can be invoiced with the credited Product again, because it keeps track of the total quantity invoiced compared to the number of items ordered (or delivered, depending on your invoicing policy).

So, next to manually creating a Credit Note that isn't linked to anything, you have this more "history-friendly" option to reduce the originally (ordered) Quantity to reflect the amount that is actually invoicable in total - which may sound counter intuitive, but ultimately this is what the customer is paying for.*

In general, this behavior stems from the fact that Odoo doesn't know why you've created a Credit Note - it could just have been a premature or in any other way incorrect Invoice, thus allowing you to still fully Invoice what ever was on the original Sale Order.

Order of 2 Products:

The confirmed Regular Invoice for that Sale Order:

The Credit Note for this Invoice:

The State of the Sale Order at this point:

The state after adjusting the (ordered) Quantity - note that Odoo does not provide an option to create an Invoice anymore:

*) Now, granted the fact that this is not an always-use-this-approach kind of a thing. There may be scenarios where you actually want to have an ordered Quantity deferring from the quantity invoiced which again would have you use your current approach of manually creating a Credit Note.