The H.264 video coding standard features diverse computational hot spots that need to be accelerated to cope with the significantly increased complexity compared to previous standards. In this paper, we propose an optimized application architecture for the H.264 encoder with reduced processing and which is suitable for application specific (reconfigurable) hardware platforms. Our proposed application architecture optimization for the computational amount of the Motion Compensation (MC) is independent of the actual hardware platform that is used for execution. For a MIPS processor we achieve an average speed-up of approx. 60x for MC. Our proposed application architecture reduces the overhead for Reconfigurable Platforms by distributing the actual hardware requirements amongst the functional blocks. This increases the amount of available reconfigurable hardware per data path (within a functional block) which leads to a 2.84x performance improvement. We evaluate our application architecture by means of four different hardware platforms.