'%20fill='%23C31D20'%20fill-rule='nonzero'%3e%3cpath%20d='M7.55269822,13.7609409%20C10.8552199,9.97176134%2014.2271223,7.63452926%2017.1034483,6.5677512%20L17.0816429,0.884990191%20C17.0593112,0.508963963%2016.8188535,0.182653682%2016.4703158,0.055395956%20C16.1217781,-0.0718617705%2015.7322263,0.0244201241%2015.4799397,0.300178022%20L12.9584465,3.07097775%20L0.368821458,17.4916392%20C0.0232151224,17.8857306%20-0.0894678188,18.437454%200.0732192079,18.9389805%20C0.235906234,19.4405071%200.649247199,19.815643%201.15754056,19.9230783%20C1.66583392,20.0305135%202.19185783,19.853926%202.53746417,19.4598346%20L7.5467513,13.7690073%20L7.55269822,13.7609409%20Z'%20id='路径'%3e%3c/path%3e%3cpath%20d='M10.6896552,12.0927463%20L15.1004089,17.5535201%20C15.4511303,17.8402432%2015.9409156,17.8973222%2016.3502521,17.6991755%20C16.7595885,17.5010287%2017.0117165,17.0848129%2016.993637,16.6370685%20L17.1034483,10.0202479%20C14.5615201,9.75868982%2012.477139,10.7130773%2010.6896552,12.0927463%20Z'%20id='路径'%3e%3c/path%3e%3c/g%3e%3crect%20id='矩形'%20x='0'%20y='0'%20width='24'%20height='24'%3e%3c/rect%3e%3c/g%3e%3c/g%3e%3c/svg%3e)
kubernetes标识增加说明
node label名称 |
作用 |
取值 |
使用组件 |
|---|---|---|---|
accelerator |
标识节点的处理芯片 |
huawei-Ascend910；huawei-Ascend310；huawei-Ascend310P |
Ascend Device Plugin |
host-arch |
标识节点的cpu架构 |
huawei-x86；huawei-arm |
Volcano |
masterselector |
标识MindX DL的管理节点 |
dls-master-node |
Volcano；hccl-controller；resilience-controller |
nodeDEnable |
节点noded启动的开关 |
on |
Volcano |
workerselector |
标识MindX DL的计算节点 |
dls-worker-node |
Ascend Device Plugin；noded；npu-exporter |
accelerator-type |
标识Atlas 训练服务器类型 |
card；module |
Ascend Device Plugin；Volcano |
servertype |
Atlas 200I Soc A1 核心板标识 |
soc |
Volcano |
huawei.com/Ascend910-Recover |
910芯片故障恢复标识 |
故障芯片ID |
Ascend Device Plugin |
huawei.com/Ascend910-NetworkUnhealthyRecover |
910芯片网络故障恢复标识 |
故障芯片ID |
Ascend Device Plugin |
node annotation名称 |
作用 |
取值 |
使用组件 |
|---|---|---|---|
noded/heartbeat |
noded的心跳，标识节点是否健康 |
string |
Volcano；noded；resilience-controller |
noded/heartbeat-interval |
noded的心跳间隔 |
string |
Volcano；noded；resilience-controller |
名称 |
作用 |
取值 |
使用组件 |
|---|---|---|---|
ring-controller.atlas |
标识atlas的pod |
ascend-910 |
Ascend Device Plugin；hccl-controller |
fault-scheduling |
任务故障重调度开关 |
grace；force；off |
Volcano；resilience-controller |
elastic-scheduling |
任务弹性调度开关 |
on；enable |
resilience-controller |
名称 |
作用 |
取值 |
使用组件 |
|---|---|---|---|
ascend.kubectl.kubernetes.io/ascend-910-configuration |
hccl-controller生成hccl.json的数据来源 |
字符串map |
Ascend Device Plugin；hccl-controller |
hccl/rankIndex |
断点续训中保持原rankId的依据。 |
[0,1000] |
Volcano；hccl-controller |
huawei.com/Ascend910 |
Ascend Device Plugin为pod分配芯片的依据 |
字符串 |
Volcano；Ascend Device Plugin |
huawei.com/AscendReal |
Ascend Device Plugin为pod实际分配芯片的记录 |
字符串 |
Volcano；Ascend Device Plugin |
huawei.com/kltDev |
kubelet为pod分配芯片的记录 |
字符串 |
Ascend Device Plugin |
predicate-time |
Ascend Device Plugin为pod分配芯片的顺序依据 |
字符串 |
Volcano；Ascend Device Plugin |
名称 |
命名空间 |
作用 |
使用组件 |
|---|---|---|---|
vcjob-fault-npu-cm |
volcano-system |
固化故障重调度内容 |
Volcano |
volcano-scheduler-configmap |
volcano-system |
volcano-scheduler的配置文件（原生） |
Volcano |
mindx-dl-deviceinfo-节点名称 |
kube-system |
Ascend Device Plugin上报的node的芯片信息 |
Volcano；Ascend Device Plugin；resilience-controller |
fault-config-任务名称 |
任务空间 |
断点续训需要的故障rankID信息 |
Volcano；elastic-agent |
rings-config-任务名称 |
任务空间 |
hccl.json内容 |
Ascend Device Plugin；hccl-controller |
