基于区块链的毕业设计Topographic Neural Networks – 地形神经网络

本文提供基于区块链的毕业设计国外最新区块链项目源码下载,包括solidity,eth,fabric等blockchain区块链,基于区块链的毕业设计Topographic Neural Networks – 地形神经网络 是一篇很好的国外资料

Topographic Neural Networks

This repository contains the source code of my Master Thesis at ETH Zurich exploring Topographic Neural Networks (TNNs). Please find the thesis and more information about TNNs in the docs folder.

Requirements

The code is tested with Python 3.7.1. All the package requirements are listed in requirements.txt. We use MuJoCo environments: please see here for installing MuJoCo and licensing.

Scripts

The runnable scripts are in the scripts folder:

  • bc.py either runs a single behavioral cloning experiment or multiple experiments under an Optuna search to optimize hyperparameters;
  • demo_viz.py runs a Dash website server for many plots and visualizations;
  • demo_save.py runs a Dash website server for saving many plots and visualizations;
  • eval_rl.py evaluates RL rollouts of TNN models;
  • expert.py collects expert trajectories;
  • sample.py collects random trajectories;
  • viz_rl.py produces RL rollouts of a TNN policy;
  • viz_scatter.py produces scatterplots of metrics against hyperparameters, across various runs;
  • viz_sem_sim.py produces the semantic similarity plots described in the thesis;
  • viz_single.py produces the single-environment plots described in the thesis;
  • viz_torso_centr.py produces the torso centrality plots.

The scripts are meant to be run in order: first expert.py, then bc.py, and finally any of the remaining scripts.

Example usage:

python bc.py --env_ids Walker2d-v2 

BASH

The bash folder provides self-contained BASH pipelines. The numerated scripts reproduce the experiments of the thesis, relying on the following pipeline:

  • scrpt_bc.sh runs bc.py on the Leonhard cluster;
  • scrpt_eval_rl_all.sh runs eval_rl.py for all the runs;
  • scrpt_expert.sh collects expert trajectories for all the environments;
  • scrpt_sample.sh collects random trajectories for all the environments;
  • scrpt_viz_rl_all.sh runs viz_rl.py for all the runs;
  • scrpt_viz_scatter.sh runs viz_scatter.py on the Leonhard cluster.

Example usage:

bash scrpt_bc.sh --use_gpu --hours 12 --env_ids Walker2d-v2 

Args

All the Python scripts can run with the provided default arguments, i.e. without specifying any argument.

expert.py uses the following arguments:

  • seed is the initialization for the random number generator;
  • env_id is the ID of the environments to test;
  • num_episodes represents how many episodes of behavioral cloning data to collect;
  • record_videos is a boolean flag indicating whether to record videos of the RL rollout;
  • environments_dir is the directory containing the trained experts and information about the environments. This argument defaults to specs;
  • results_dir is the directory where the run’s outputs will be stored.

bc.py uses the following additional arguments:

  • id is a string identifying the experiment;
  • run_type accepts the values exp or search, and tells the script whether to run a single experiment or a search;
  • env_ids is a list of IDs for the Gym environments to train and test. If the list contains more than one element, then the environments are trained in a multi-tasking fashion;
  • metric is the name of the metric to be optimized in the search. Please choose between those in METRICS_AND_DIRECTIONS in __init__.py;
  • use_scheduler overwrites the homonymous parameter in the config file, telling whether to use a learning rate scheduler;
  • use_early_stopping overwrites the homonymous parameter in the config file, telling whether to use early stopping;
  • lr overwrites the homonymous parameter in the config file, defining the learning rate;
  • baseline is a boolean flag indicating whether to train a baseline model instead of a TNN;
  • val_rl is a boolean flag indicating whether to run RL rollouts every validation epoch;
  • no_test_rl is a boolean flag indicating that RL rollouts should not be performed at the end of training;
  • num_epochs is the number of training epochs;
  • num_trajectories is the number of expert trajectories to train on;
  • num_trials is the number of trials of the hyperparameter search;
  • grid_init accepts either rand or skel, and indicates whether the neurons’ positions should be initialized at random or at the joints’ positions;
  • params_dir is the directory containing the configuration files. This argument defaults to params, where you can find example hyperparameters’ configurations;
  • environments_dir is the directory containing information about the environments. This argument defaults to specs;
  • results_dir is the directory where the run’s outputs will be stored.

viz_rl.py uses the following additional arguments:

  • results_base_dir is the experiment directory to use for experiments and visualizations.

viz_sem_sim.py uses the following additional arguments:

  • runs_dirs is a list of experiment directoriers to use for experiments and visualiztaions;
  • runs_index is a list of identification names for the directories in runs_dirs.

The arguments used in the remaining scripts all have the same meaning as described above.

Some BASH scripts accept arguments in the same form as the Python scripts. Please refer to the args_*.py scripts for more information about the arguments accepted. Leonhard-specific arguments include:

  • use_gpu is a boolean flag indicating whether to use GPU on Leonhard;
  • email is a boolean flag indicating whether the Leonhard job should send emails at its start and end;
  • hours is the hour limit of the Leonhard job.

Params

The scripts read hyperparameters from JSON configuration files. You can find example files in the params folder, which is divided into three subfolders:

  • files in exp are used when run_type is exp;
  • files in search are used when run_type is search.

Specs

The specs folder contains information about the environments used in our experiments. Each environment has its specific folder, e.g. for Walker2d-v2:

  • Walker2d-v2_actions.json stores, for each entry of the actions’ vector, semantic information about the body part executing the action;
  • Walker2d-v2_combs.json stores the list of combinations for the semantic attributes;
  • Walker2d-v2_obs.json stores, for each entry of the observations’ vector, semantic information about the body part to which the observation refers;
  • Walker2d-v2_coords.json stores, for each joint, 2D quantized quadrant coordinates.

We collect joints coordinates only for the following agents: Ant-v2, HalfCheetah-v2, Hopper-v2, Humanoid-v2, and Walker2d-v2.


地形神经网络

这个存储库包含了我在苏黎世ETH探索地形神经网络(TNNs)的硕士论文的源代码。请在docs文件夹中找到论文和关于TNNs的更多信息

需求

代码是用python3.7.1测试的。所有的软件包要求都列在requirements.txt中。我们使用MuJoCo环境:请参阅此处了解MuJoCo的安装和许可

脚本

可运行的脚本位于scripts文件夹中:

  • bc.py在Optuna搜索下运行单个行为克隆实验或多个实验以优化超参数
  • demo_viz.py运行一个Dash网站服务器,用于许多绘图和可视化
  • demou save.py运行一个Dash网站服务器,用于保存许多绘图和可视化效果
  • evalu rl.py评估TNN模型的rl展开
  • expert.py收集专家轨迹
  • sample.py收集随机轨迹
  • 即py生成TNN策略的rl卷展
  • vizèu scatter.py生成跨各种运行的超参数度量散点图
  • 即semu sim.py生成本文描述的语义相似图
  • vizèu single.py生成本文描述的单一环境图
  • viz_torso_centr.py生成躯干中心性图

这些脚本的运行顺序是:首先是expert.py,然后是bc.py,最后是任何剩余的脚本

示例用法:

python bc.py --env_ids Walker2d-v2 

BASH

bash文件夹提供自包含的bash管道。经过计算的脚本根据以下管道重现了本文的实验:

  • scrpt_bc.sh在Leonhard集群上运行bc.py
  • scrptu evalu rlu all.sh为所有运行运行evalu rl.py
  • scrptu expert.sh收集所有环境的专家轨迹
  • 所有运行的scrptèvizèrlèu all.sh运行vizèrl.py
  • scrptèvizèu scatter.sh在Leonhard集群上运行vizèu scatter.py
  • 种子是随机数发生器的初始化

示例用法:

bash scrpt_bc.sh --use_gpu --hours 12 --env_ids Walker2d-v2 

Args

所有Python脚本都可以使用提供的默认参数运行,即不指定任何参数

expert.py使用下列参数:

  • 环境id是要测试的环境的id
  • numu sessions表示要收集多少集的行为克隆数据
  • 录制视频是一个布尔标志,指示是否录制RL卷展栏的视频
  • 环境目录是包含经过培训的专家和有关环境的信息的目录。此参数默认为specs
  • resultsu dir是存储运行输出的目录
  • id是标识实验的字符串

bc.py使用下列附加参数:

  • runu type接受exp或search值,并告诉脚本是运行单个实验还是搜索
  • metric是要在搜索中优化的度量的名称。请在度量中的u和u init.py中的u方向中进行选择
  • useu scheduler覆盖配置文件中的同名参数,告知是否使用学习率调度器
  • useu earlyu stopping覆盖配置文件中的同名参数,告知是否使用early stopping
  • lr覆盖配置文件中的同名参数,定义学习率
  • 基线是一个布尔标志,指示是否训练基线模型而不是TNN
  • valu rl是一个布尔标志,指示是否在每个验证历元运行rl卷展栏
  • nou testu rl是一个布尔标志,表示在训练结束时不应执行rl卷展
  • numu epochs是训练次数
  • numèu trajections是要训练的专家轨迹数
  • numu trials是超参数搜索的试验数
  • paramsu dir是包含配置文件的目录。此参数默认为params,您可以在其中找到示例超参数的配置
  • 环境u dir是包含环境信息的目录。此参数默认为specs
  • resultsu dir是存储运行输出的目录
  • resultsu baseu dir是用于实验和可视化的实验目录
  • 运行索引是运行目录中目录的标识名列表
  • 使用gpu是一个布尔标志,指示是否在硬件上使用gpu

vizèrl.py使用下列附加参数:

  • email是一个布尔标志,指示作业是否应该在开始和结束时发送电子邮件

vizèsemèsim.py使用下列附加参数:

  • exp中的文件在runu type为exp时使用
  • 当“运行类型”为“搜索”时,将使用搜索中的文件

其余脚本中使用的参数都具有与上述相同的含义

一些BASH脚本接受与Python脚本相同形式的参数。有关接受的参数的详细信息,请参阅args_u3;*.py脚本。Leonhard特定的参数包括:

  • Walker2d-v2u actions.json为动作向量的每个条目存储执行动作的身体部位的语义信息
  • Walker2d-v2u combs.json存储语义属性的组合列表
  • Walker2d-v2u obs.json为观察向量的每个条目存储关于观察所指身体部位的语义信息

参数

脚本从JSON配置文件读取超参数。您可以在params文件夹中找到示例文件,该文件夹分为三个子文件夹:

  • Walker2d-v2_coords.json为每个关节存储二维量化象限坐标
  • files in search are used when run_type is search.

规范

specs文件夹包含有关实验中使用的环境的信息。每个环境都有其特定的文件夹,例如对于Walker2d-v2:

  • Walker2d-v2_actions.json stores, for each entry of the actions’ vector, semantic information about the body part executing the action;
  • Walker2d-v2_combs.json stores the list of combinations for the semantic attributes;
  • Walker2d-v2_obs.json stores, for each entry of the observations’ vector, semantic information about the body part to which the observation refers;
  • Walker2d-v2_coords.json stores, for each joint, 2D quantized quadrant coordinates.

我们仅为以下代理收集关节坐标:Ant-v2、HalfCheetah-v2、Hopper-v2、Humanoid-v2和Walker2d-v2

部分转自网络,侵权联系删除区块链源码网

www.interchains.cc

https://www.interchains.cc/23060.html

区块链毕设网(www.interchains.cc)全网最靠谱的原创区块链毕设代做网站 部分资料来自网络,侵权联系删除! 最全最大的区块链源码站 ! QQ3039046426
区块链知识分享网, 以太坊dapp资源网, 区块链教程, fabric教程下载, 区块链书籍下载, 区块链资料下载, 区块链视频教程下载, 区块链基础教程, 区块链入门教程, 区块链资源 » 基于区块链的毕业设计Topographic Neural Networks – 地形神经网络

提供最优质的资源集合

立即查看 了解详情