Deprecated: Assigning the return value of new by reference is deprecated in /www/htdocs/w00b2db9/cake/dispatcher.php on line 157

Deprecated: Assigning the return value of new by reference is deprecated in /www/htdocs/w00b2db9/cake/dispatcher.php on line 221

Deprecated: Assigning the return value of new by reference is deprecated in /www/htdocs/w00b2db9/cake/libs/controller/controller.php on line 308

Deprecated: Assigning the return value of new by reference is deprecated in /www/htdocs/w00b2db9/cake/libs/controller/controller.php on line 347

Deprecated: Assigning the return value of new by reference is deprecated in /www/htdocs/w00b2db9/cake/libs/controller/controller.php on line 535

Deprecated: Assigning the return value of new by reference is deprecated in /www/htdocs/w00b2db9/cake/libs/controller/controller.php on line 805

Deprecated: Assigning the return value of new by reference is deprecated in /www/htdocs/w00b2db9/cake/libs/controller/component.php on line 125

Deprecated: Assigning the return value of new by reference is deprecated in /www/htdocs/w00b2db9/cake/libs/view/view.php on line 681

Deprecated: Assigning the return value of new by reference is deprecated in /www/htdocs/w00b2db9/cake/libs/class_registry.php on line 55

Strict Standards: Non-static method Inflector::camelize() should not be called statically, assuming $this from incompatible context in /www/htdocs/w00b2db9/cake/dispatcher.php on line 89

Strict Standards: Non-static method Configure::getInstance() should not be called statically in /www/htdocs/w00b2db9/cake/basics.php on line 275

Strict Standards: Non-static method Inflector::underscore() should not be called statically in /www/htdocs/w00b2db9/cake/basics.php on line 289

Strict Standards: Non-static method Inflector::underscore() should not be called statically, assuming $this from incompatible context in /www/htdocs/w00b2db9/cake/libs/controller/controller.php on line 243

Strict Standards: Non-static method Inflector::classify() should not be called statically, assuming $this from incompatible context in /www/htdocs/w00b2db9/cake/libs/controller/controller.php on line 246

Strict Standards: Non-static method Inflector::camelize() should not be called statically, assuming $this from incompatible context in /www/htdocs/w00b2db9/cake/libs/inflector.php on line 385

Strict Standards: Non-static method Inflector::singularize() should not be called statically, assuming $this from incompatible context in /www/htdocs/w00b2db9/cake/libs/inflector.php on line 385

Strict Standards: Non-static method Inflector::getInstance() should not be called statically, assuming $this from incompatible context in /www/htdocs/w00b2db9/cake/libs/inflector.php on line 301

Strict Standards: Non-static method Set::pushDiff() should not be called statically, assuming $this from incompatible context in /www/htdocs/w00b2db9/cake/libs/inflector.php on line 286

Strict Standards: Non-static method Set::pushDiff() should not be called statically, assuming $this from incompatible context in /www/htdocs/w00b2db9/cake/libs/inflector.php on line 287

Strict Standards: Non-static method Set::pushDiff() should not be called statically, assuming $this from incompatible context in /www/htdocs/w00b2db9/cake/libs/inflector.php on line 288

Strict Standards: Non-static method Inflector::underscore() should not be called statically, assuming $this from incompatible context in /www/htdocs/w00b2db9/cake/libs/controller/controller.php on line 247

Strict Standards: Non-static method Configure::getInstance() should not be called statically in /www/htdocs/w00b2db9/cake/basics.php on line 418

Strict Standards: Non-static method Inflector::underscore() should not be called statically in /www/htdocs/w00b2db9/cake/basics.php on line 425

Strict Standards: Non-static method Security::inactiveMins() should not be called statically, assuming $this from incompatible context in /www/htdocs/w00b2db9/cake/libs/session.php on line 145

Warning: session_start(): Cannot send session cookie - headers already sent by (output started at /www/htdocs/w00b2db9/cake/dispatcher.php:157) in /www/htdocs/w00b2db9/cake/libs/session.php on line 154

Warning: session_start(): Cannot send session cache limiter - headers already sent (output started at /www/htdocs/w00b2db9/cake/dispatcher.php:157) in /www/htdocs/w00b2db9/cake/libs/session.php on line 154

Warning: Cannot modify header information - headers already sent by (output started at /www/htdocs/w00b2db9/cake/dispatcher.php:157) in /www/htdocs/w00b2db9/cake/libs/session.php on line 155

Strict Standards: Non-static method Set::extract() should not be called statically, assuming $this from incompatible context in /www/htdocs/w00b2db9/cake/libs/session.php on line 291

Strict Standards: Non-static method Set::insert() should not be called statically, assuming $this from incompatible context in /www/htdocs/w00b2db9/cake/libs/session.php on line 360

Strict Standards: Non-static method Set::extract() should not be called statically, assuming $this from incompatible context in /www/htdocs/w00b2db9/cake/libs/session.php on line 361

Strict Standards: Non-static method Set::insert() should not be called statically, assuming $this from incompatible context in /www/htdocs/w00b2db9/cake/libs/session.php on line 360

Strict Standards: Non-static method Set::extract() should not be called statically, assuming $this from incompatible context in /www/htdocs/w00b2db9/cake/libs/session.php on line 361

Strict Standards: Non-static method Set::insert() should not be called statically, assuming $this from incompatible context in /www/htdocs/w00b2db9/cake/libs/session.php on line 360

Strict Standards: Non-static method Set::extract() should not be called statically, assuming $this from incompatible context in /www/htdocs/w00b2db9/cake/libs/session.php on line 361

Strict Standards: Non-static method ClassRegistry::addObject() should not be called statically, assuming $this from incompatible context in /www/htdocs/w00b2db9/cake/libs/view/view.php on line 225

Strict Standards: Non-static method ClassRegistry::getInstance() should not be called statically, assuming $this from incompatible context in /www/htdocs/w00b2db9/cake/libs/class_registry.php on line 66

Strict Standards: Non-static method Inflector::underscore() should not be called statically, assuming $this from incompatible context in /www/htdocs/w00b2db9/cake/libs/view/view.php on line 483

Strict Standards: Non-static method Configure::getInstance() should not be called statically, assuming $this from incompatible context in /www/htdocs/w00b2db9/cake/libs/view/view.php on line 500

Strict Standards: Non-static method Configure::getInstance() should not be called statically in /www/htdocs/w00b2db9/cake/basics.php on line 362

Strict Standards: Non-static method Inflector::underscore() should not be called statically in /www/htdocs/w00b2db9/cake/basics.php on line 369

Strict Standards: Non-static method Inflector::variable() should not be called statically, assuming $this from incompatible context in /www/htdocs/w00b2db9/cake/libs/view/view.php on line 679

Strict Standards: Non-static method Inflector::camelize() should not be called statically, assuming $this from incompatible context in /www/htdocs/w00b2db9/cake/libs/inflector.php on line 395

Strict Standards: Non-static method Inflector::underscore() should not be called statically, assuming $this from incompatible context in /www/htdocs/w00b2db9/cake/libs/inflector.php on line 395

Strict Standards: Non-static method Configure::getInstance() should not be called statically in /www/htdocs/w00b2db9/cake/basics.php on line 362

Strict Standards: Non-static method Inflector::underscore() should not be called statically in /www/htdocs/w00b2db9/cake/basics.php on line 369

Strict Standards: Non-static method Inflector::variable() should not be called statically, assuming $this from incompatible context in /www/htdocs/w00b2db9/cake/libs/view/view.php on line 679

Strict Standards: Non-static method Inflector::camelize() should not be called statically, assuming $this from incompatible context in /www/htdocs/w00b2db9/cake/libs/inflector.php on line 395

Strict Standards: Non-static method Inflector::underscore() should not be called statically, assuming $this from incompatible context in /www/htdocs/w00b2db9/cake/libs/inflector.php on line 395

Strict Standards: Non-static method Configure::getInstance() should not be called statically in /www/htdocs/w00b2db9/cake/basics.php on line 362

Strict Standards: Non-static method Inflector::underscore() should not be called statically in /www/htdocs/w00b2db9/cake/basics.php on line 369

Strict Standards: Non-static method Inflector::variable() should not be called statically, assuming $this from incompatible context in /www/htdocs/w00b2db9/cake/libs/view/view.php on line 679

Strict Standards: Non-static method Inflector::camelize() should not be called statically, assuming $this from incompatible context in /www/htdocs/w00b2db9/cake/libs/inflector.php on line 395

Strict Standards: Non-static method Inflector::underscore() should not be called statically, assuming $this from incompatible context in /www/htdocs/w00b2db9/cake/libs/inflector.php on line 395

Strict Standards: Non-static method Configure::getInstance() should not be called statically in /www/htdocs/w00b2db9/cake/basics.php on line 362

Strict Standards: Non-static method Inflector::underscore() should not be called statically in /www/htdocs/w00b2db9/cake/basics.php on line 369

Strict Standards: Non-static method Inflector::variable() should not be called statically, assuming $this from incompatible context in /www/htdocs/w00b2db9/cake/libs/view/view.php on line 679

Strict Standards: Non-static method Inflector::camelize() should not be called statically, assuming $this from incompatible context in /www/htdocs/w00b2db9/cake/libs/inflector.php on line 395

Strict Standards: Non-static method Inflector::underscore() should not be called statically, assuming $this from incompatible context in /www/htdocs/w00b2db9/cake/libs/inflector.php on line 395

Strict Standards: Non-static method Inflector::variable() should not be called statically, assuming $this from incompatible context in /www/htdocs/w00b2db9/cake/libs/view/view.php on line 679

Strict Standards: Non-static method Inflector::camelize() should not be called statically, assuming $this from incompatible context in /www/htdocs/w00b2db9/cake/libs/inflector.php on line 395

Strict Standards: Non-static method Inflector::underscore() should not be called statically, assuming $this from incompatible context in /www/htdocs/w00b2db9/cake/libs/inflector.php on line 395

Strict Standards: Non-static method Configure::getInstance() should not be called statically, assuming $this from incompatible context in /www/htdocs/w00b2db9/cake/libs/view/view.php on line 538

Strict Standards: Non-static method Configure::read() should not be called statically, assuming $this from incompatible context in /www/htdocs/w00b2db9/cake/libs/view/view.php on line 405

Strict Standards: Non-static method Configure::getInstance() should not be called statically, assuming $this from incompatible context in /www/htdocs/w00b2db9/cake/libs/configure.php on line 154

Strict Standards: Non-static method Configure::read() should not be called statically, assuming $this from incompatible context in /www/htdocs/w00b2db9/cake/libs/view/view.php on line 597

Strict Standards: Non-static method Configure::getInstance() should not be called statically, assuming $this from incompatible context in /www/htdocs/w00b2db9/cake/libs/configure.php on line 154
PyBrain

Strict Standards: Non-static method Configure::read() should not be called statically, assuming $this from incompatible context in /www/htdocs/w00b2db9/cake/libs/view/view.php on line 597

Strict Standards: Non-static method Configure::getInstance() should not be called statically, assuming $this from incompatible context in /www/htdocs/w00b2db9/cake/libs/configure.php on line 154

Features

Videos

This video presentation was shown at the ICML Workshop for Open Source ML Software on June 25, 2010. It explains some of the features and algorithms of PyBrain and gives tutorials on how to install and use PyBrain for different tasks.



This video shows some of the learning features in PyBrain in action.

Algorithms

We implemented many useful standard and advanced algorithms in PyBrain, and in some cases created interfaces to existing libraries (e.g. LIBSVM). This list shows some of the methods you can use with PyBrain.

Supervised Learning

  • Back-Propagation
  • R-Prop
  • Support-Vector-Machines (LIBSVM interface)
  • Evolino

Unsupervised Learning

  • K-Means Clustering
  • PCA/pPCA
  • LSH for Hamming and Euclidean Spaces
  • Deep Belief Networks

Reinforcement Learning

Value-based

  • Q-Learning (with/without eligibility traces)
  • SARSA
  • Neural Fitted Q-iteration

Policy Gradients

  • REINFORCE
  • Natural Actor-Critic

Exploration Methods

  • Epsilon-Greedy Exploration (discrete)
  • Boltzmann Exploration (discrete)
  • Gaussian Exploration (continuous)
  • State-Dependent Exploration (continuous)

Black-box Optimization

  • (Stochastic) Hill-climbing
  • Particle Swarm Optimization (PSO)
  • Evolution Strategies (ES)
  • Covariance Matrix Adaption ES (CMA-ES)
  • Natural Evolution Strategies (NES)
  • Fitness Expectation-Maximization (FEM)
  • Finite Difference Gradient Descent
  • Policy Gradients with Parameter Exploration (PGPE)
  • Simultaneous Perturbation Stochastic Approximation (SPSA)
  • Genetic Algorithms (GA)
  • Competitive Co-Evolution
  • (Inner/Inverse) Memetic Search
  • Multi-Objective Optimization NSGA-II

Networks

PyBrain has an emphasis on network architectures, which can be trained and manipulated with almost all of the algorithms offered by PyBrain. Here are some examples of available networks and network architectures:
  • Feed-forward networks, including Deep Belief Networks and Restricted Boltzmann Machines (RBM)
  • Recurrent networks (RNN), including Long Short-Term Memory (LSTM) architectures
  • Multi-Dimensional Recurrent Networks (MDRNN)
  • Self-Organizing Maps (Kohonen networks)
  • Reservoirs
  • Bidirectional networks
  • Custom-designed topologies

Tools

  • Plotting / Visualization
  • netCDF support
  • XML read/write
  • Variety of different dataset classes

Environments

Environments in PyBrain are scenarios or test cases, in which an agent can be trained or an algorithm can be tested. Environments share a common interface, thus making it very easy to switch agents. PyBrain comes with a view predefined most basic environments:

Toy Environments

  • ShipSteering
  • Mazes (MDP/POMDP)

3D Environments

These environments have additional dependencies (e.g. OpenGL, ODE) that require extra software. More documentation can be found in the respective environment folders.
  • In the FlexCube Environment, twelve synchronous, continuously controllable edges of a virtual cube provide a complex continuous action space which is rendered with OpenGL.
  • The ODE Environment contains the ODE physics engine and allows arbitrary objects to interact with themself and other objects in a pysically realistic way. The scene can be rendered with OpenGL to give the user a precise information about what is going on in the scenario.
  • Johnnie (a robot in the ODE environment)
  • PencilBalancer (another example in the ODE environment)

Games

  • CaptureGame
  • Pente
  • Gomoku

Benchmarks

Function Environments

  • Unimodal
  • Multimodal
  • Multi-Objective

Pole-Balancing

  • Single-Pole with different tasks
  • Double-Pole

Wrappers for external benchmarks

  • Car-Racing
  • RLGlue

If you have programmed your own environment and think others might find it useful, please contact us and let us know. We are looking forward to add your environment to the list.


Strict Standards: Non-static method Configure::read() should not be called statically in /www/htdocs/w00b2db9/app/webroot/index.php on line 84

Strict Standards: Non-static method Configure::getInstance() should not be called statically in /www/htdocs/w00b2db9/cake/libs/configure.php on line 154