How to reset goals when picking or pushing objects

[gautica]

Hi,
thanks for the interesting work. I am not sure how you set the goal for the pickup or pushing task, because the goal is the position of the object. As I understand, the goal of all the failure trials before touching the object is set to be the same object position, as it doesn't move. But it doesn't sound very correct to me. Is it set in this way? Thank you very much

[TianhongDai]

@gautica In goal-oriented DRL, we have two types of goals: {"achieved goals", "desired goal"}. In "PickAndPlace" and "Pushing" tasks, the desired goal position is the final goal position, which is indicated by the red spot in demo GIF. At the beginning of each episode, the desired goal is randomly sampled from the pre-defined goal space; The achieved goal is the position of the object (the cube in these two tasks.). The aim of these tasks is to control the robotic arm to manipulate the object close to the desired goal position (i.e. dist(achieved goal, desired goal) < threshold). Hope this answer helps you.

[123gzy]

请问 在换任务时，直接改任务名称吗，改了任务名称之后发现在push等其他任务成功率为0，还是需要调参数或者更改其他内容吗？

[123gzy]

When changing the task name, do I need to change the task name directly, and find that the success rate of other tasks such as push is 0 after changing the task name?

[TianhongDai]

@123gzy 请问你可以给我一下你现在用的环境名称吗，包括gym环境以及mujoco的版本。我今天晚上自己测试一下。

[123gzy]

哇 ！我昨天试了试可以了 就用下面的命令行，可能不同任务，难易程度不一样，线程数不太一样，epoch数目也不同。所以导致我直接运行没什么效果。 感谢你的回消息 太感谢了 你写的代码也特别好！对我的帮助很大！如果方便的话还有一个请教的地方是，如果在一个任务下训练的模型，可以直接迁移到另一个任务吗，我最近在搞元强化学习，不知道是要改任务的环境吗，环境来自gym,这些问题有点头大。

…

------------------&nbsp;原始邮件&nbsp;------------------ 发件人: "TianhongDai/hindsight-experience-replay" ***@***.***&gt;; 发送时间:&nbsp;2024年11月7日(星期四) 晚上9:25 ***@***.***&gt;; ***@***.******@***.***&gt;; 主题:&nbsp;Re: [TianhongDai/hindsight-experience-replay] How to reset goals when picking or pushing objects (Issue #30) @123gzy 请问你可以给我一下你现在用的环境名称吗，包括gym环境以及mujoco的版本。我今天晚上自己测试一下。 — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.Message ID: ***@***.***&gt;

[123gzy]

@123gzy请问你可以给我一下你现在使用的环境名称吗，包括gym环境以及mujoco的版本。我今天晚上自己测试一下。

哇 ！我昨天试了试可以了 就用下面的命令行，可能不同任务，难易程度不一样，线程数不太一样，epoch数目也不同。所以导致我直接运行没什么效果。 感谢你的回消息 太感谢了 你写的代码也特别好！对我的帮助很大！如果方便的话还有一个请教的地方是，如果在一个任务下训练的模型，可以直接迁移到另一个任务吗，我最近在搞元强化学习，不知道是要改任务的环境吗，环境来自gym,这些问题有点头大。

[TianhongDai]

@123gzy 直接迁移可能有点困难，但是可以参考一下metaworld以及它相关的一些工作: https://arxiv.org/pdf/1910.10897