Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Status keeps PENDING while training #4

Open
Fishermanykx opened this issue Jun 2, 2023 · 0 comments
Open

Status keeps PENDING while training #4

Fishermanykx opened this issue Jun 2, 2023 · 0 comments

Comments

@Fishermanykx
Copy link

Hello, I set up the environment as the README says and do the training, but the trainer keeps in PENDING status and does not require any CPU or GPU resources like the image shown below. Is the code complete enough to reproduce the result shown in your paper? Is there some way to solve this problem? I'm waiting for your answer.

image

This is my train.yaml

local_dir: "./ray_results"

env_config: {
        FILENAME: /home/fisher/data/sim/goseek-allscenes-v0.1.4.x86_64,
        SCENES: [1, 2, 3, 4],
        episode_length: 400,
        video_log_path: "./videos",
        collision_reward: 0,
        target_found_reward: 1,
        success_dist: 2,
        no_collisions: False,
        modalities: ["RGB_LEFT", "SEGMENTATION", "DEPTH"],
        visible_targets: False,
        use_esdf_data: True,
        esdf_data: {'1': /home/fisher/data/esdf/scene_1_esdf_slices.npy,
                    '2': /home/fisher/data/esdf/scene_2_esdf_slices.npy,
                    '3': /home/fisher/data/esdf/scene_3_esdf_slices.npy,
                    '4': /home/fisher/data/esdf/scene_4_esdf_slices.npy,
                    },
        scene_graph: {"1": /home/fisher/data/dsgs/dsg_dict_scene_1.pkl,
                      "2": /home/fisher/data/dsgs/dsg_dict_scene_2.pkl,
                      "3": /home/fisher/data/dsgs/dsg_dict_scene_3.pkl,
                      "4": /home/fisher/data/dsgs/dsg_dict_scene_4.pkl,
                    },
    }

model: {
        custom_model: gnn_actor_critic,
        custom_model_config: {
            fields: {
                nodes: GRAPH_NODES,
                edges: GRAPH_EDGES,
                node_shapes: GRAPH_NODE_SHAPE,
                edge_shapes: GRAPH_EDGE_SHAPE,
            },
            in_features: 10,
            graph_conv_features: [64, 64, 64],
            n_layer_nodes: 8,
            embedding_size: [512],
        },
        max_seq_len: 400,
        vf_share_layers : false
    }

# training
lr : 1.0e-4
gamma : 0.99
lambda : .9
vf_loss_coeff : 0.25
clip_param : 0.3
vf_clip_param : 25
entropy_coeff : 0.01

# batch
train_batch_size : 1024 # 4096
sgd_minibatch_size : 128
num_sgd_iter : 30
rollout_fragment_length: 400

# env
env : goseek
framework : torch
num_workers : 0
num_gpus : 1
num_envs_per_worker : 1
num_gpus_per_worker: 0.25
log_level : INFO

ckpt_save_freq_timesteps: 10000

evaluation_interval : 50
evaluation_num_episodes : 50
evaluation_num_workers: 1
evaluation_config: {
    env_config: {
            SCENES: [5],
            RANK: 3,
            FILENAME: /home/fisher/data/sim/goseek-allscenes-v0.1.4.x86_64,
            esdf_data: {'5': /home/fisher/data/esdf/scene_5_esdf_slices.npy},
            scene_graph: {"5": /home/fisher/data/dsgs/dsg_dict_scene_5.pkl},
        }
    }
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant