Neural Networking Biological Features

ANN Biological features implemented by BAI 2006+

See ANNsource-2a4d-24October2009a.zip for historic source record of all biological neural network features implemented by BAI before 18 February 2009:

  • Competitive reinforcement learning ("-idata [string]: neural network experience data set file" / "THtestANNusingCombatExperiences")
    cf. TD-gammon (Tesauro 1995), OpenAI/Dota (Bansal et al 2017), AlphaGo (Silver et al 2017), AlphaZero (Silver et al 2018).
  • Depth ("-layers [float]: neural network formation number of top level layers")
    cf. deep learning (e.g. Krizhevsky et al 2012).
  • Convolution ("-divtype [float]: subnet divergence type.. 6: diverging square 2D radial bias")
    cf. pattern/character recognition (Fukushima 1980), object recognition (LeCun et al 1999).
  • Segregated connections ("-con [float]: probability of a neuron having connection to a previous layer neuron")
    cf. dropout (Srivastava et al 2014), sparsity (Ahmad & Luiz Scheinkman 2019).
  • Direct connections ("conall [float]: probability of a neuron having direct link with all previous layers neurons")
    cf. shortcut/skip layer connection history (e.g. Bishop 1995, Ripley 1996, Venables & Ripley 1999, Intrator & Intrator 2001, Curry & Morgan 2003, etc), Inception (Szegedy et al 2015), Highway Networks (Srivastava et al 2015), Resnet (Zhang et al 2016), Densenet (Huang et al 2017).
  • Subnetting ("-usesubnets: create subnets")
    cf. ensembles (e.g. He et al 2015, Tao 2019)
  • Classification Network
    cf. Gated Linear Networks/Predictive Coding
  • Self-supervised training via True Positive vs Random Negative Sampling
    cf. Contrastive Learning