LMSAdaptiveFilterVsRLQ-LearningAlgorithm

Amit Choudhary
December 12, 2011

Attach:README.txt Attach:LMS_Program Attach:Q-Learning_Program

The work discuss comparison between two different algorithms Least Mean Square(LMS) and Q-Learning in the reinforcement learning field which look very similar in first glance. The similarity is with respect to training and update of algorithm parameters based on current experience. In the project we have tested both of the algorithms using same input data under few varieties and compared their behavior to understand what makes them different. The obvious similarity turns out to be misleading and this paper is focused to clearing this misconception with some experimental results.

Concepts Demonstrated

Q-Learning(Original Version) AI technique has been explored in this Project. The Project talks about its limitations in continuous domain and compare it with Least Mean Square technique. One-step prediction system has been designed to analyze each algorithms for random input data.

Innovation

The innovative cut to this project comes from the fact that this is the first attempt to do a comparison of Q-Learning with an algorithms from unrelated field.

Evaluation of Results

With the experiments we conclude that Q-Learning and LMS are two different algorithms not only in their method of solving the problem but also the kind of problem they are capable of solving it. The ability of Q-Learning to store past actions and choose the best among many when in a particular state makes it best suited in robot motion planning. Similarly, the ability to track input and suit your filter parameters accordingly makes LMS an ideal choice for task like noise cancellations.