VideosThis video presentation was shown at the ICML Workshop for Open Source ML Software on June 25, 2010. It explains some of the features and algorithms of PyBrain and gives tutorials on how to install and use PyBrain for different tasks.
This video shows some of the learning features in PyBrain in action.
AlgorithmsWe implemented many useful standard and advanced algorithms in PyBrain, and in some cases created interfaces to existing libraries (e.g. LIBSVM). This list shows some of the methods you can use with PyBrain.
- Support-Vector-Machines (LIBSVM interface)
- K-Means Clustering
- LSH for Hamming and Euclidean Spaces
- Deep Belief Networks
- Q-Learning (with/without eligibility traces)
- Neural Fitted Q-iteration
- Natural Actor-Critic
- Epsilon-Greedy Exploration (discrete)
- Boltzmann Exploration (discrete)
- Gaussian Exploration (continuous)
- State-Dependent Exploration (continuous)
- (Stochastic) Hill-climbing
- Particle Swarm Optimization (PSO)
- Evolution Strategies (ES)
- Covariance Matrix Adaption ES (CMA-ES)
- Natural Evolution Strategies (NES)
- Fitness Expectation-Maximization (FEM)
- Finite Difference Gradient Descent
- Policy Gradients with Parameter Exploration (PGPE)
- Simultaneous Perturbation Stochastic Approximation (SPSA)
- Genetic Algorithms (GA)
- Competitive Co-Evolution
- (Inner/Inverse) Memetic Search
- Multi-Objective Optimization NSGA-II
NetworksPyBrain has an emphasis on network architectures, which can be trained and manipulated with almost all of the algorithms offered by PyBrain. Here are some examples of available networks and network architectures:
- Feed-forward networks, including Deep Belief Networks and Restricted Boltzmann Machines (RBM)
- Recurrent networks (RNN), including Long Short-Term Memory (LSTM) architectures
- Multi-Dimensional Recurrent Networks (MDRNN)
- Self-Organizing Maps (Kohonen networks)
- Bidirectional networks
- Custom-designed topologies
- Plotting / Visualization
- netCDF support
- XML read/write
- Variety of different dataset classes
EnvironmentsEnvironments in PyBrain are scenarios or test cases, in which an agent can be trained or an algorithm can be tested. Environments share a common interface, thus making it very easy to switch agents. PyBrain comes with a view predefined most basic environments:
- Mazes (MDP/POMDP)
3D EnvironmentsThese environments have additional dependencies (e.g. OpenGL, ODE) that require extra software. More documentation can be found in the respective environment folders.
- In the FlexCube Environment, twelve synchronous, continuously controllable edges of a virtual cube provide a complex continuous action space which is rendered with OpenGL.
- The ODE Environment contains the ODE physics engine and allows arbitrary objects to interact with themself and other objects in a pysically realistic way. The scene can be rendered with OpenGL to give the user a precise information about what is going on in the scenario.
- Johnnie (a robot in the ODE environment)
- PencilBalancer (another example in the ODE environment)
- Single-Pole with different tasks
Wrappers for external benchmarks
If you have programmed your own environment and think others might find it useful, please contact us and let us know. We are looking forward to add your environment to the list.