Run error

[leyan88]

I am an anylogic beginner, download your code on GitHub, open the model in anylogic, but the model error:

[leyan88]

If it is convenient, I would like you to add a video using the explanatory model. As a beginner, it is a little difficult for me to learn from the documentation.

[leyan88]

Thank you very much for your prompt response! I appreciate you taking the time to get back to me so quickly. I apologize for my delayed reply; and I wanted to make sure I had enough time to properly review your project before responding.

I'm excited to hear that you are open to collaborating on integrating your project with AnyLogic. I believe that combining your work with AnyLogic could lead to some innovative applications, especially in the realm of decision-making and simulation.

Could you please provide more details on whether your project can be adapted to train DQN or DPG agents?

[MarcEscandell]

Since alpyperl is integrated with Open AI Gymnasium framework, you can use any RL model that is compatible. For example, if you use rllib (the recommended package, as alpyperl offers evaluation/deployment functionalities) here's a list of all available models.

In case of DQN, you can use the following python script for training (as an example):

from alpyperl import AnyLogicEnv
from ray.rllib.algorithms.dqn import DQNConfig


# Set checkpoint directory.
checkpoint_dir = "./resources/trained_policies/cartpole_v0"

# Initialize policy.
policy = (
    DQNConfig()
    .env_runners(
        num_env_runners=2,
        num_envs_per_env_runner=2
    )
    .fault_tolerance(
        recreate_failed_env_runners=True,
        num_consecutive_env_runner_failures_tolerance=3
    )
    .environment(
        AnyLogicEnv, 
        env_config={
            'run_exported_model': True,
            'exported_model_loc': './resources/exported_models/cartpole_v0',
            'show_terminals': False,
            'verbose': False,
            'checkpoint_dir': checkpoint_dir,
            'env_params': {
                'cartMass': 1.0,
                'poleMass': 0.1,
                'poleLength': 0.5,
            }
        }
    )
    .build()
)

# Perform training.
for _ in range(100):
    result = policy.train()

# Save policy checkpoint.
policy.save(checkpoint_dir)
print(f"Checkpoint saved in directory '{checkpoint_dir}'")

# Close all enviornments.
# NOTE: This is required to be called for correct checkpoint saving by ALPypeRL.
policy.stop()

[leyan88]

Thank you for your answers! I also want to know if ALpypeRL supports multi-agent reinforcement learning?

[MarcEscandell]

Hei! ALPypeRL does not support multi-agent RL at the moment. I have considered it as the next feature to include, but I haven't got a timeline. I'll keep you updated if I do so :)

[leyan88]

Thank you very much! This is a feature I am looking forward to and much needed！