From Diablo Wiki
Brawling is a form of consensual PvP combat. In a brawl each character consents to engage in head to head combat. Brawls can be one on one or may take place in duelling games, which are FFA (free for all) battles with multiple characters engaged at once.
Brawling was introduced Patch 1.0.7 on the Patch Test Realm released in January 2013. During its testing, based on community feedback, it was changed from Duelling to Brawling to better reflect the nature of the fight.
There's a lot of power in a name, and we think that the idea of a brawl more accurately describes the system we created.
 Brawling is not PK
Diablo III does not allow Player Killing.
 Brawling Rules
Regular PvP players generally adhere to a list of (informal, largely unwritten) rules. These are generally lists of banned practices, or buggy items that must not be used since they convey an unfair or unbeatable advantage. What is on and off this ever-changing list is not known to outsiders. It's therefore a good idea to read some duelling forums or talk to other players before a fight, to be sure of the ground rules.
 Duelling in Diablo II
In Diablo II, duels were customarily fought in areas that were cleared of monsters. Which character builds were most effective varied a great deal between patches, and there was seldom any "best" character, since there was too much variety in strengths and weaknesses. Rock-paper-scissors was the way of it, though some rocks were necessarily heavier than others.
A recent guest article on Diii.net discussed how duelling has evolved over time Diablo II, and it's a useful read if you want a more historical perspective.