lose one’s rag

Definition

Slang

  • verb to lose one’s temper, lose control of oneself. This mainly working-class expression is of obscure origin; the word rag has meant variously one’s tongue, a flag, to tease and to bluster or rage, but none of these senses can be definitively linked to the modern phrase.
http://www.dictionarycentral.com/definition/lose-one-s-rag.html