In this work we present the first fully three-dimensional image reconstruction scheme for optical tomography that is based on the equation of radiative transfer. This scheme builds on the previously introduced concept of model-based iterative image reconstruction, in which a forward model provides prediction of detector readings, and a gradient-based updating scheme minimizes an objective function, which is defined as the difference between predicted and measured data. The forward model is solved by using an even-parity approach to reduce the time-independent radiative transfer equation to an elliptic self-adjoint equation of second order. This equation is discretized using a finite element method, in which we apply a preconditioned conjugate gradient method with a multigrid-based preconditioner to solve the arising linear algebraic system. The gradient of the objective function is found by employing an adjoint differentiation method to the forward solver. Initial tests on synthetic data have shown robustness and good convergence of the algorithm.