A toggle button is a button which stays down when clicked once and goes up when clicked a second time.
Use a toggle button to indicate a state.
Example: A word processor should use toggle buttons to indicate the state of "Bold", "Italic" or "Underline" formatting.
Do not use a toggle button to indicate an action.
Example: A music player should not use a toggle button to implement a Play/Pause button. It should use a normal button and adjust the icon and label to represent the action which would be performed when clicked.
Label of toggle buttons can often be reduced to a "noun only" form instead of the classic "verb (+ noun)" used with classic buttons. The "noun only" form should be favored over the "verb (+ noun)" form whenever it does not cause any ambiguity as this form reduces clutter.
Example: A toggle button to show/hide a sidebar can be labeled either "Show Sidebar" or simply "Sidebar".
Never change the icon or label of a toggle button. A toggle button is very similar to a check box and should thus behave in the same way. The fact that it is pressed down is enough to indicate the current state of the element it represents.
This means that in the previous example, "Show Sidebar" should not be turned into "Hide Sidebar" when the sidebar is visible. If you consider it more explicit to change the icon and label of your button then you should use a normal button, not a toggle button.