This is now resolved as of the forum upgrade on May 26, 2021. You should now be able to remove categories with the X in your user preferences.
Original post:
I may have found a workaround for this. If you go to the category list and then choose one of the categories you are watching, I think you can remove it from the “Watching First Post” list there.
