Rethinking Innateness: A Connectionist Perspective on Development by Jeffrey L. Elman, Elizabeth A. Bates, Mark H. Johnson, Annette Karmiloff-Smith, Domenico Parisi, and Kim Plunkett. MIT Press, 1996 Text
Exercises in Rethinking Innateness: A Handbook for Connectionist Simulations Kim Plunkett and Jeffrey L. Elman. MIT Press, 1997. Handbook
You won't need to buy either, but especially the latter book provides useful insights into the simulator and the PDP approach to connectionist modeling. A User's Manual for the simulator will be distributed in class.
The assignment is to explore the capabilities of some very simple network architectures to learn simple functions. You are to build networks using Tlearn and to describe their behavior. The first task is the logical AND function: AND(0,0)=AND(0,1)=AND(1,0)=0; AND(1,1)=1.
First experiment with a network that directly connects the two input units to the one output unit. Tlearn allows you to change a number of parameters such as learning rate, momentum, etc. See if you can can find settings that will enable Tlearn to learn AND essentially every time. For what range of settings will it learn 75% of the time. Then try learning the same AND function starting with a net that has two intermediate units between the input and output, but no direct input/output links. How much does this help?
The second task is the logical SAME function: SAME(0,0)=SAME(1,1)=1; SAME(0,1)=SAME(1,0)=0.
Perform a similar set of experiments using networks without and with two extra "hidden layer" units. Explain why the results are quite different in this case.
You should turn a record (hand written is fine) of the parameters that you tried on each example and what happened.
This assignment is due in class on Thursday, September 18 or by earlier electronic submission.