To improve the reconstructed image quality with a given number of sampling points, nonuniform sampling is desired which adapts the sampling density according to the local bandwidth of the signal. Determination of optimal sampling positions and interpolation from nonuniform samples through the use of a coordinate mapping which converts nonuniform samples into points on a regular sampling lattice. We then introduce a nonuniform sampling scheme which embeds the samples in a generally deformed mesh structure that can be easily mapped to a regular sampling lattice. The optimal samples or the mesh is generated by minimizing the interpolation error. The numerical difficulty associated with dealing with nonuniform samples are circumvented by mapping all the operations to the master domain where the samples are uniformly distributed. With this scheme, in order to maintain the mesh topology, unnecessary nodes are usually allocated in large but smooth regions. For an improved sampling efficiency, a hierarchical nonuniform sampling scheme is also developed. Which embeds the samples in a generalized quadtree structure. Compared to its nonhierarchical counterpart, this scheme can reduce the numbers of samples significantly, under the same visual quality constraint.