Only use exclamation points (exclamation marks) in games.
In general, exclamation points should be used sparingly in user-interface strings. In particular, they should not be used for error messages, to avoid an effect upon the user similar to WRITING IN ALL-CAPS.
The main exception to this is games, where one might write "Congratulations! You have won!". But, even here, if the game could not be saved to disk, the error message would not take an exclamation mark.