Tip: If you're serious about improving your writing, one of the best ways is to improve your vocabulary — not so you know how to use big words, but so you can choose the right word at the right time to convey your meaning precisely and simply. To improve your vocabulary, we highly recommend you try the popular vocabulary-improvement software called Ultimate Vocabulary. Click Ultimate Vocabulary for details.*
This common English error comes about through small slip ups made in spoken English, which then get transferred to written English.
The correct usage is could have, must have, should have, and would have. That's it.
I should have gone to that lecture, but I played with the ducks instead. I could have done the right thing, but I didn't. I would have gone if I'd known they were going to give out free jelly snakes! I must have been mad to have skipped class!
Where the error comes in is that people often make a contraction of words in spoken English, slurring two words into one another and dropping the occasional letter here and there. The ha from have gets lost in this instance. So could have ends up as could've, would have becomes would've, must have is shortened to must've, and should have becomes should've.
Now, some sounds in English are very similar to one another! F and V are two such sounds. 'VE and OF sound very much the same. So could've sounds quite a lot like could'of, would've sounds like would'of, must've sounds like must'of, and should've sounds like should'of. These then get expanded to could of, would of, must of, and should of. All of which are incorrect.
So remember you HAVE to HAVE the HAVE and you won't get tripped up!
*We have agreed to receive a commission from some sales of Ultimate Vocabulary because we are happy to endorse this award-winning vocabulary-improvement software.