Cooperative diversity envisages to provide improved quality of service at the physical layer, with the assumption that wireless nodes are always willing to cooperate. This assumption may be valid in networks where cooperation is confined to nodes whose intentions are known a priori. However, in commercial wireless networks where nodes may misbehave for malicious or selfish intentions, it is difficult to maintain stable cooperation without a mechanism to detect and mitigate effects of misbehavior. In this paper we present a distributed mechanism to detect misbehaving nodes in a cooperative network which is characterized by a minimum frame success rate requirement and hybrid automatic repeat request (HARQ) is implemented to ensure reliable communication. The detection mechanism is developed by observing a number of first transmission attempts to learn how many frames can be transmitted successfully with zero retransmissions (first HARQ phase). Probability of successful transmission for the first transmission attempts is then estimated and its percentage deviation from the minimum frame success rate is measured to detect absence or presence of misbehavior. We show that the proposed mechanism can effectively detect selfish and malicious nodes. The detection mechanism can also be integrated into a reputation based scheme where a source node inform other nodes about the behavior of its partner.