The Interfacial Transition Zone (ITZ) refers to the progressive change in the physical properties of mortar occurring in the vicinity of the aggregates. The most common cause of concrete failure is the accumulation of microcracks, which initiate at the ITZ. In this work, an attempt is made to investigate the ITZ and its mechanical properties. The current work examines the influence of the ITZ on the tensile response of a representative volume element (RVE) of concrete. Microscopic tensile stresses are upscaled to the macroscopic scale, and the material’s constitutive relationship is determined for spherical aggregates embedded in the mortar matrix. A model is proposed for the tensile response of RVE under uniaxial tensile loading. The microscopic stresses are calculated using a homogenization approach with Eshelby solutions. As the microcracks propagate, the Elastic Modulus and Poisson’s ratio of the different phases decreases, leading to pre-peak non-linear behavior. The model results are further extended to comment on the number and size of microcracks generated in the specimen. At the failure, 23.61% increase in the tensile stress of ITZ is observed when compared with macroscopic tensile stresses of concrete. The microcracks are observed to initiate in the ITZ at 58% of peak load. A parametric study is conducted to investigate the impact of aggregate size, aggregate content, and ITZ properties on the stress–strain behavior of RVE.