RL Glue and Codecshttp://mloss.orgUpdates and additions to RL Glue and CodecsenMon, 12 Oct 2009 07:50:03 -0000RL Glue and Codecs -- Glue 3.x and Codecs<html><p>Please go to for information on how to cite RL-Glue if you are using in your own research and publications. </p> <p>RL-Glue allows agents, environments, and experiments written in Java, C/C++, Matlab, Python, and Lisp to inter operate, accelerating research by promoting software re-use in the community. </p> <p>Note: This is a release candidate, not a final release. We are actively soliciting feedback from the community about any problems with the software or documentation that can be improved before a final release in January 31 2009. </p> <p>Update posted: Dec 10/2008. Some updates to the protocol, notably removing the seed/key methods. This should be the last release candidate before the final release on January 31 2009. </p> <p>Update posted: Oct 11/2008, Big change for main RL-Glue and C-Codec to use const-pointers instead of structs by value in parameters and return types. Breaks backward compatibility. See the tech manual in docs. </p> <p>Update posted: Oct 8/2008, fixed memory leak in RL-Glue, fixed skeleton experiment build on Linux, updated some Cygwin compatibility stuff. </p> <p>Overview <hr /> Inspired by related psychological theory, in computer science, reinforcement learning is a sub-area of machine learning concerned with how an agent ought to take actions in an environment so as to maximize some notion of long-term reward. Reinforcement learning algorithms attempt to find a policy that maps states of the world to the actions the agent ought to take in those states... -- Wikipedia Reinforcement Learning Article </p> <p>RL-Glue is a set of common guidelines for the reinforcement learning community to follow to allow us to share and compare agents and environments with greater ease. The software implementation of RL-Glue is the reusable glue interface to connect the basic parts of a learning experiment. </p> <p>RL-Glue supports interaction between agents, environments, and experiment programs in two different modes. In direct-compile mode, all three modules are written in C/C++ and compiled together into a single executable program. </p> <p>In the more flexible socket mode: agents, environments, and experiments use inter-process communication through sockets, either locally on one computer or over the network or Internet. In socket mode, agents, environments, and experiments written in a variety of languages can interact with each other transparently. The language-specific software that allows creations from a particular language to connect to RL-Glue is called a codec. We currently have codecs for: </p> <ul> <li> C/C++ </li> <li> Java </li> <li> Python </li> <li> Matlab </li> <li> Lisp </li> </ul> <p>Members of the reinforcement learning community are welcome to write their own language-or-project specific codecs to use with RL-Glue. The Lisp codec is an example of a user-contributed codec. There are currently codecs in development to connect projects as diverse as: a real-time strategy game, an atari emulator, and a robot to RL-Glue. </p> <p>The RL-Glue software project, combined with the RL-Glue codecs are a powerful tool that allows members of the reinforcement learning community to re-use each others agents, environments, and experiment programs to help quicken the pace of research. Before RL-Glue most researchers implemented their own experiment protocol, making collaboration difficult. </p> <p>RL-Glue has been the base for the last few reinforcement learning competitions, and that trend will continue with the 2009 Reinforcement Learning Competition. </p> <p>What's new in RL-Glue 3.0 <hr /> - A new homepage: </p> <ul> <li><p>Revamped build system (autotools) for maximum cross-platform install compatibility (Linux, Unix, MacOS, Cygwin) -Installing has never been simpler: &gt;$ ./configure &gt;$ make &gt;$ sudo make install </p> </li> <li><p>RL-Glue now installs to /usr/local </p> <ul> <li> Headers and libs in standard search paths<ul> <li> Compiling agents/environments/experiments has never been easier: &gt;$ gcc MyAgent.c -lrlagent -o myAgent.exe </li> </ul> </li> </ul> </li> <li><p>Codecs for C/C++, Java, Python, MATLAB AND LISP &lt;--- MATLAB AND LISP! </p> </li> <li><p>charArray (String) observation and action types! </p> </li> <li><p>Documentation </p> <ul> <li> General RL-Glue Overview </li> <li> RL-Glue Technical Manual </li> <li> A manual for each codec! </li> </ul> </li> </ul> <p>History of RL-Glue <hr /> We can trace RL-Glue back as far as 1996 through a project by Rich Sutton and Juan Carlos Santamaria called RL-Interface. Since then, the project has gone through several designs and languages. Over time the objectives of the project became more ambitious - it grew from being a convenient calling convention within a single language to a complete protocol allowing all sorts of various languages to communicate with each other. </p></html>Brian Tanner, Adam White, Richard S. SuttonMon, 12 Oct 2009 07:50:03 -0000 learningnips2008