In this article an adaptive Constrained Finite Time Optimal (CFTO) controller design is presented for unknown, discrete-time linear systems, whose dynamics are identified via Set-Membership Identification (SMI) algorithms. The controller consists of two modules: a) the Set Membership(SM)-Identifier that identifies the system's dynamics through the extended Recursive Least Squares (eRLS) while providing the vertices of an orthotope that contains the nominal identification parameter vector, and b) the CFTO-controller; the tuning of the controller is based on a batch tuning procedure. The controller's feasible operating region remains unchanged for: a) all SM-orthotopes that are subsets of the last orthotope for which the controller's tuning was based, and b) the calculated CFTO-controller remains constant for a large period of time. In the opposite case the control partition of the controller is re-calculated (batch process) based on the current vertices of the identified SM-orthotope. Simulation results are presented that prove the efficacy and the validity of the suggested control scheme.