We propose a novel method for object tracking and segmentation by using online Hough forests and convex relaxation. Our method extracts object contour during tracking rather than using a bounding box or an ellipse to locate the object. Unlike conventional active contour methods that use consistent intensity or color distribution as constraints, our method uses Hough forests for online discriminative learning, resulting in faster convergence and more accurate segmentation. We use Bayesian formulation to model the probability of the contour, given the description of the regions and the edges. Additionally, the Hough forests provide an estimate of the initial location of the object to improve accuracy. Segmentation is then formulated as a convex relaxation optimization problem. Experimental results show the effectiveness and robustness of our method. The results also show that our method outperforms some of the state-of-the-art methods.