My code samples of Asa H 2.0 light (blogs of 10 Feb. 2011 and 14 May 2012) are for a single layer in the hierarchy. These layers are then connected one to another by something like the code in my 26 Aug. 2013 blog (That blog assumes layers write to or read from data files. This I've done if I want to keep a record of these calculation steps and the categories being formed. More direct connection is possible, of course, and I've done that too.)
But how many layers are needed? I've used as many as at least 9 layers in various experiments to date. (I may have used more than 9, I'd have to look back and see. I recall using at least 9 layers on occasion.) The first or second layer might detect edges, for example, the next might detect lines at various orientations, the next might sense corners, the next might detect simple shapes, the next might detect eyes, or other features, then faces, then complete creatures, etc. etc. How deep should deep-learning be?
Similarly, how big a pattern can each layer learn? (For example, what should TMAX be set to in the code from my 10 Feb. 2011 blog?) In many of my experiments TMAX=5 was used. Possibly TMAX=10 would be better. In more advanced experiments I have sometimes let TMAX grow during a run.
These questions are related to the issue of what curriculum should be provided as training for Asa. What should an AI learn (ANY AI, any machine learning algorithm), and in what order? This was less of a problem if the AI is intended to operate in a limited domain. The question becomes more important if the AI is to operate in the "real world."
No comments:
Post a Comment