Unmanned Aerial Vehicles (UAVs), better known as drones, have significantly advanced fields such as aerial surveillance, military reconnaissance, cadastral surveying, disaster monitoring, and delivery services. However, UAVs rely on civilian (unauthenticated) GPS for navigation which can be trivially spoofed. In this paper, we present DeepSIM, a satellite imagery matching approach to detect GPS spoofing attacks against UAVs based on deep learning. We make use of the camera(s) a typical UAV is equipped with, and present a system that compares historical satellite images of its GPS-based position (spaceborne photography) with real-time aerial images from its cameras (airborne imagery). Historical images are taken from, e. g., Google Earth or NASA WorldWind. To detect GPS spoofing attacks, we investigate different deep neural network models that compare the real-time camera images with the historical satellite images. To train and test the models, we have constructed the SatUAV dataset (consisting of 967 image pairs), partially by using real UAVs such as the DJI Phantom 4 Advanced. Real-world experimental results show that our best model has a success rate of about 95% in detecting GPS spoofing attacks within less than 100 milliseconds. Our approach does not require any modification of the existing GPS infrastructures and relies only on public satellite imagery, making it a practical solution for many everyday scenarios.