The OP stated he wished to be charged when he flew, not on the departure date of the ticket.
Over booking is there to allow people more flexibility to fly. An airline over books a flight because a certain percentage of people no-show the flights. Since a portion of the tickets are refundable, if they do not show, the airline does not get paid. Business people need the flexibility of refundable/changeable tickets. Secondly, while the cheaper tickets are non-refundable they are changeable for a fee. Since the airline overbooks the flight of 100 seats and sells lets say 120 seats on the flight, it is offering an extra 20 people the opportunity to fly on a flight that is convenient for them. If the airline did not do that, 20 people would have had to choose a different flight and miss a plane that in all likely hood had 20 open seats at departure.
At AA, they used very sophisticated tracking software (IIRC, the software was used by several other airline since it was so effective) to track each flights history. They are quite effective in this and very few people are turned away due to over booking. It does happen but not very often and maybe by 1 or 2 people if it does happen.
Bottom line is it offers a wider selection of flights to the passenger and it is done because they know that people like your boss exist.
I suppose that paying at dept is a possibility but I equate it to a pre-paid part. If the dealer does not have a part and they need to special order it, I have to pay up front to ensure I come and get it and pay for it. I suspect it has more to do with tracking and logistics as much as anything else. Easier to get the money up front and not have to wait till the airline gets a head count and process the ticket. Plus, given all the changes made to a ticket and the fees involved, that would be a huge head ache for the airlines.
I know when AA instituted the bag fees, they took effect for tickets issued after a specific date. I also know several of the other majors did the same. I worked for AA at the time and we got emails about the fees.
As for lost bags, if you knew what goes on to get your bag from A to B you would be amazed that any of them get there. Not sure where you got the 10% figure from but according to this
site, the number is not even 1%. For Feb of 2010, the number is 4.01 bags mishandled per 1000 bags.