Project details for RLPy

Logo RLPy 1.3a

by bobklein2 - August 28, 2014, 14:34:35 CET [ Project Homepage BibTeX Download ]

view ( today), download ( today ), 0 subscriptions


RLPy is a framework for performing sequential decision making experiments in Python. RLPy provides a fine-grained view of learning agents, breaking them into modular components and providing a library for each. Additionally, RLPy provides a wide variety of problem domains to test these agents - these are listed at bottom.

Parallelization: Easily scale experiments by running them in parallel on multiple cores of a single machine (user only needs to specify the number of cores) or on HTCondor computing cluster.

Hyperparameter Optimization: Built-in support for optimizing hyperparameters with state-of-the-art methods (using the hyperopt package). The user only needs to specify the parameters and their bounds.

Code Profiling: Easily identify performance bottlenecks of the code with built-in profiling support. A color-coded call graph of execution reveils slow functions.

Plotting: User specifies experimental configuration and number of runs for statistical significance. Then using tool, user need only specify quantities to appear on the graph; runs of the same configuration are automatically associated and averaged, and various configurations can be plotted simultaneously with confidence intervals.

Learning Agent Components:

Value Function Representations:

  • Bellman Error Basis Functions
  • Fourier Basis Functions
  • Incremental Feature Dependency Discovery (iFDD)
  • Orthogonal Matching Pursuit - Temporal Difference (OMP-TD)
  • Radial Basis Functions
  • Tabular
  • Tile Coding

Exploration Policies:

  • Epsilon-Greedy
  • Fixed
  • Gibbs
  • Uniform Random

Learning Algorithms:

  • Greedy-GQ
  • Least-Squares Policy Iteration
  • Natural Actor-Critic
  • Policy Iteration
  • Q-Learning
  • Trajectory-based Value Iteration
  • Value Iteration

Problem Domains:

  • Acrobot
  • Bicycle Balancing
  • BlocksWorld
  • CartPole Balancing (2-state or 4-state)
  • CartPole Swingup (2-state or 4-state)
  • Fifty-State ChainMDP
  • FlipBoard
  • GridWorld
  • HIV Treatment
  • Helicopter Hovering
  • Intruder Monitoring
  • MountainCar
  • MultiTrack
  • Pac-Man
  • Persistent Search and Track ("PST")
  • Pinball
  • PuddleWorld
  • RC Car
  • Swimmer
  • System Administrator
Changes to previous version:
  • Fixed bug where results using same random seed were different with visualization turned on/off
  • Created RLPy package on pypi (Available at
  • Switched from custom logger class to python default
  • Added unit tests
  • Code readability improvements (formatting, variable names/ordering)
  • Restructured TD Learning heirarchy
  • Updated tutorials
BibTeX Entry: Download
Supported Operating Systems: Platform Independent
Data Formats: Any
Tags: Python, Scalable, Reinforcement Learning Library, Modular, Parallelizable
Archive: download here


No one has posted any comments yet. Perhaps you'd like to be the first?

Leave a comment

You must be logged in to post comments.