10-11 周三 shell xargs tr curl 做大事情

最近发现，shell的小工具非常的强大，简单记录下

tr命令

-d 删除字符串1中所有输入字符。
-s 删除所有重复出现字符序列，只保留第一个；即将重复出现字符串压缩为一个字符串

-d 用于删除查询到的字符串中的空格。

[test3@NH-DC-NM134-C05-YW-HPC-27 bin]$ cat extract 
#!/bin/bash
# 从easybuild配置文件中提取指定键的值cat /home/test3/.config/easybuild/config.cfg | grep $1 | awk -F "=" '{print $2}' | tr -d " "

xargs

curl命令发起多个POST请求

发起POST请求，拼接到URL中

cat images | xargs -I {} -P 3 curl http://10.101.14.36:48001/api/fast-build/task/check-image?image_name={} -X POST -d ‘’

JSON请求，请求体放在文件中

find ./ -name raw_json | xargs -p -n1 -li curl -X POST -H “Content-Type: application/json” -d @raw_json http://10.101.14.36:48001/api/fast-build/task/build-image

其中raw_json内容如下:

{"webSSHSecret": "j7yC3VEVGFAJIRmnel+XVA==", "jupyterLabSecret": "qkrhxQmWOe5kvpJpplvTuQ==", "task_data": {"task_name": "10.101.12.129-lf99fl@zhejianglab.com-1694483700", "target_image_name": "10.101.12.129/lf99fl-zhejianglab.com/lftestbuild-test:1", "callback_url": "http://pre.alkaidos.cn/api/app/dros-ic-platform/harbor/image/callback"}, "dockerfile_json": {"base_image": "10.101.12.129/base/centos:8-test", "maintainer": "1676883790847406082", "image_installer_config": {"python_env": {"present": "", "update": true, "target": "Python-3.7.14.tgz", "install_loc": "/usr/local/dros/python"}, "pip_installer_config": {"installer_name": "pip", "install": {"present": "", "update": true, "target": "get-pip.py", "install_loc": ""}, "source": {"installer_name": "", "type": "ali", "file_name": "pip.conf"}, "software_list": [], "delimiter": "", "python_version": []}, "package_manager_installer_config": {"installer_name": "yum", "install": {"present": "4.4.2", "update": false, "target": "", "install_loc": ""}, "source": {"installer_name": "", "type": "", "file_name": ""}, "software_list": [], "delimiter": "", "python_version": []}, "conda_installer_config": {"installer_name": "conda", "install": {"present": "", "update": true, "target": "Anaconda3-5.3.1-Linux-x86_64.sh", "install_loc": ""}, "source": {"installer_name": "", "type": "qinghua", "file_name": ".condarc"}, "software_list": [], "delimiter": "", "python_version": []}, "webSSHSecret": "", "jupyterLabSecret": ""}}}

多进程触发下载

[test3@NH-DC-NM134-C05-YW-HPC-27 bin]$ cat downdep 
#!/bin/bash
#$1 为将待下载的依赖列表所在的文件名，使用eb -Dr *.eb >dep生成
count=`cat $1 | grep "\[ \]" | wc -l`cat $1 | grep "\[ \]" | cut -d "(" -f 1 | awk -F "/" '{print $NF}' |xargs -I file -P "${count}" sh -c "eb --fetch file" 2 > down.error