Extracting higher-level relationships in connectionist models

Connectionist networks excel at extracting statistical regularities but have trouble extracting higher-order relationships. Clark and Thornton suggest that a solution to this problem might come from Elman (1993), but I argue that the success of Elman's single recurrent network is illusory, and we show that it cannot in fact represent abstract relationships that can be generalized to novel instances, undermining Clark and Thornton's key arguments.