End-to-End Training of Deep Visuomotor Policies · HackerPapers discussions

End-to-End Training of Deep Visuomotor Policies — mentioned by WilliamDhalgren

Thu, 25 May 2017 12:36:48 GMT

> AlphaGo essentially baked good movies into value and policy network by playing millions of times. I don't think that's a very good description of how AlphaGo was trained at all; you're essentially saying it merely overfits the training set, yet it clearly generalizes rather well to unseen board situations and still evaluates them sucessfully. No machine learning system would be found usefull if all it could do is merely memorize the training data. Re the use of deep reinforcement learning, well for one the role of reinforcement learning in the first version of AlphaGo, the one described in…

Paper and discussion history. A mention is not an endorsement.

End-to-End Training of Deep Visuomotor Policies — mentioned by deepnet

Tue, 03 May 2016 23:51:18 GMT

A Hinton inspired speculation about a Robot that plans & communicates. I posit that general A.I. must be embodied to develop, it must learn from the real world by interacting with it and learn by trial and error. Deep learning is a huge leap for AI. It learns best from raw data. Babies learn motor control & input interpretation first[1]. Moravec's paradox proposes learning these basics are the hard foundation that higher levels of reason build on. Abeel and Levine's work[2] where robots learn robust functions to map from pixels to motor torques for domestic tasks show solutions built on…

Paper and discussion history. A mention is not an endorsement.

End-to-End Training of Deep Visuomotor Policies — mentioned by deepnet

Thu, 28 Apr 2016 22:30:10 GMT

The 1st A.I. winter was 1973 when Donald Michie's work was eviscerated by the Lighthill report. Edinburgh University, under Michie, was leading the world in A.I. Michie, Ambler & Popplestone's 1969 Freddy Robot could assemble scattered bricks using A.I. vision & GOF symbolic AI. Michie (& Chambers) BOXES algorithm was succesfully learning to balance poles using reinforcement learning, from raw-sensor data outputting motor torques, real robotics trained end-to-end not simulated. Edinburgh had a rival LISP known as POPLOG (Popplestone & Burstal) used UK & Europe wide. BOXES was akin to todays…

Paper and discussion history. A mention is not an endorsement.

End-to-End Training of Deep Visuomotor Policies — mentioned by jpfr

Wed, 16 Dec 2015 16:15:26 GMT

Prototypical case of the 80/20 rule. He has implemented the happy case. But that system is nothing people realistically would want to drive their cars. What he did is impressive. But the results are not that outlandish for a talented person. 1) Hook up a computer to the CAN-Bus network of the car [1] and attach a bunch of sensor peripherals. 2) Drive around for some time and record everything to disk. 3) Implement some of the recent ideas from deep reinforcement learing [2,3]. For training, feed the system with the oberservations from test drives and reward actions that mimick the reactions…

Paper and discussion history. A mention is not an endorsement.

End-to-End Training of Deep Visuomotor Policies — mentioned by zhanwei

Fri, 23 Oct 2015 02:24:22 GMT

It seems to me to be all about having the right prior and planning for exploration. Policy search methods (http://arxiv.org/abs/1504.00702) assume that there aren't many trajectories that make sense (based on prior knowledge/testing in simulator) and search for the best ones among those that make sense using real-world data. Even within policy search you need some kind of exploration such injecting gaussian noise in trajectories. The hard part is to come up with a model for exploration.

Paper and discussion history. A mention is not an endorsement.

End-to-End Training of Deep Visuomotor Policies — mentioned by davmre

Fri, 23 Oct 2015 00:24:46 GMT

The point is that supervised training on large networks requires millions or billions of examples. In RL, that means potentially billions of execution trajectories, which can be really expensive if you're executing on an actual, physical robot. And even then, supervised classification objectives are more informative than just a scalar reward: when a supervised system makes a wrong prediction you don't just tell it "sorry, that's wrong", you also tell it what the right prediction would have been. That extra information is really valuable for learning. In the black-box RL setting, you only see…

Paper and discussion history. A mention is not an endorsement.

End-to-End Training of Deep Visuomotor Policies — HN submission

Tue, 07 Jul 2015 03:32:43 GMT

End-To-End Training of Deep Visuomotor Policies

6 points; 1 discussion reply on HN. Paper and source history. A submission is not an endorsement.

End-to-End Training of Deep Visuomotor Policies — HN submission

Tue, 26 May 2015 12:17:25 GMT

End-to-End Training of Deep Visuomotor Policies [pdf]

5 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

End-to-End Training of Deep Visuomotor Policies — mentioned by nl

Fri, 22 May 2015 12:51:18 GMT

I don't know enough robotics to follow the paper[1] entirely, but this seems the relevant part for your question: We evaluated our method by training policies for hanging a coat hanger on a clothes rack, inserting a block into a shape sorting cube, fitting the claw of a toy hammer under a nail with various grasps, and screwing on a bottle cap. The cost function for these tasks encourages low distance between three points on the end-effector and corresponding target points, low torques, and, for the bottle task, spinning the wrist. The equations for these cost functions follow prior work.[23]…

Paper and discussion history. A mention is not an endorsement.

End-to-End Training of Deep Visuomotor Policies — mentioned by deepnet

Fri, 22 May 2015 08:44:41 GMT

Learning motor torques directly from vision is a very important result. https://youtu.be/EtMyH_--vnU This talk by Sergey Levine, Pieter Abbeel's PostDoc outlines Berkley's end-to-end deep-training visuomotor control in detail. Here is the paper : End-to-End Training of Deep Visuomotor Policies, Sergey Levine, Chelsea Finn, Trevor Darrell, Pieter Abbeel. http://arxiv.org/abs/1504.00702

Paper and discussion history. A mention is not an endorsement.