### Abstract

One test of a new training algorithm is how well the algorithm generalizes from the training data to the test data. It is shown that a new training algorithm termed double backpropagation improves generalization by simultaneously minimizing the normal energy term found in backpropagation and an additional energy term that is related to the sum of the squares of the input derivatives (gradients). In normal backpropagation training, minimizing the energy function tends to push the input gradient to zero. However, this is not always possible. Double backpropagation explicitly pushes the input gradients to zero, making the minimum broader, and increases the generalization on the test data. The authors show the improvement over normal backpropagation on four candidate architectures and a training set of 320 handwritten numbers and a test set of size 180.

Original language | English (US) |
---|---|

Title of host publication | Proceedings. IJCNN - International Joint Conference on Neural Networks |

Editors | Anon |

Publisher | Publ by IEEE |

Pages | 145-150 |

Number of pages | 6 |

ISBN (Print) | 0780301641 |

State | Published - 1992 |

Event | International Joint Conference on Neural Networks - IJCNN-91-Seattle - Seattle, WA, USA Duration: Jul 8 1991 → Jul 12 1991 |

### Publication series

Name | Proceedings. IJCNN - International Joint Conference on Neural Networks |
---|

### Other

Other | International Joint Conference on Neural Networks - IJCNN-91-Seattle |
---|---|

City | Seattle, WA, USA |

Period | 7/8/91 → 7/12/91 |

### ASJC Scopus subject areas

- Engineering(all)

## Fingerprint Dive into the research topics of 'Double backpropagation increasing generalization performance'. Together they form a unique fingerprint.

## Cite this

*Proceedings. IJCNN - International Joint Conference on Neural Networks*(pp. 145-150). (Proceedings. IJCNN - International Joint Conference on Neural Networks). Publ by IEEE.