The FlexCube Environment

State of the art reinforcement learning methods deal with complex, only partial observable and noisy problems. Implementing new ways of reinforcement learning and trying to compare different learning methods, requires a framework that is adequate complex. On the other hand the framework must be fast, so that comparing methods can be done just in time.

The FlexCube Environment provides that: Twelve synchronous, continuous controllable edges provide a complex continuous action space. The spring morphology provides a noisy, partial observable also continuous state space.


The framework is a test case for all methods that should be capable of handling partial observable Markov decision processes (POMDPs) with mapping from continuous state- to continuous action space. But despite of that principal complexity, the framework is still fast and easy to handle.

Physics and Structure

Passive physics

Random behaviour
Body: masspoints connected with springs
Powers: gravitation and inertia
Friction: total friction