This paper focuses on the optimal control of continuous-time linear time-varying uncertain systems with completely unknown internal dynamics and proposes a novel approach which leads to an optimal controller with guaranteed stability. A model-free algorithm of adaptive dynamic programming is employed to deal with the uncertainty of system parameters, yielding an optimal feedback controller for the system subject to a predefined cost. Then the stability of the system in time-varying uncertain situation which may undergo parameter changes or jumps is analyzed from the perspective of finite-time stability. On the basis of these results, a switching control strategy is presented to ensure the stability of the time-varying uncertain system with desired optimality properties in the long run. The effectiveness of the strategy is verified by simulations on a DC torque motor servo system.