Log in as admin
Banners > New Banner
Fill in banner text
Press "Create banner"
Follow "New Banner"
Repeat steps 3-5
The second banner replaces the first, which is as it should be. But it does not actually change the first banner to inactive, i.e. if you look at https://test.archiveofourown.org/admin/banners, both have their "Active" button looking pressed, and if you follow the link to edit the first banner, the "Active" box is still checked.
The first banner should be deactivated.
Editing the first banner does not make it replace the second one, so this isn't particularly dangerous, just confusing.