gitlab bash

Few things are more satisfying to me than one elegant line of Bash that automates hours of tedious work.

没有什么比让Bash自动完成数小时繁琐工作的Bash优雅系列令我满意的了。

As part of some recent explorations into automatically re-creating my laptop with Bash scripts (post to come!), I wanted to find a way to easily clone my GitHub-hosted repositories to a new machine. After a bit of digging around, I wrote a one-liner that did just that.

作为最近使用Bash脚本自动重新创建笔记本电脑的探索的一部分，我想找到一种轻松地将GitHub托管的存储库克隆到新计算机上的方法。经过一番摸索之后，我写了一篇这样的单线纸。

Then, in the spirit of not putting all our eggs in the same basket, I wrote another one-liner to automatically create and push to GitLab-hosted backups as well. Here they are.

然后，本着不把所有鸡蛋都放在同一篮子的精神，我写了另一种单行代码来自动创建并推送到GitLab托管的备份。他们来了。

一个Bash一线式克隆您所有的GitHub存储库 (A Bash one-liner to clone all your GitHub repositories)

Caveat: you’ll need a list of the GitHub repositories you want to clone. The good thing about that is it gives you full agency to choose just the repositories you want on your machine, instead of going in whole-hog.

注意：您将需要要克隆的GitHub存储库的列表。这样做的好处是，它使您可以完全选择仅要在计算机上存储的存储库，而不必花很多精力。

You can easily clone GitHub repositories without entering your password each time by using HTTPS with your 15-minute cached credentials or, my preferred method, by connecting to GitHub with SSH. For brevity I’ll assume we’re going with the latter, and our SSH keys are set up.

您可以轻松地克隆GitHub存储库，而无需每次都输入HTTPS并使用15分钟的缓存凭据，或者，我首选的方法是使用SSH连接到GitHub，而无需每次输入密码。为简便起见，我假设我们要使用后者，并且我们已经设置了SSH密钥。

Given a list of GitHub URLs in the file gh-repos.txt, like this:

在文件gh-repos.txt中给出GitHub URL的列表，如下所示：

git@github.com:username/first-repository.git
git@github.com:username/second-repository.git
git@github.com:username/third-repository.git

We run:

我们跑：

xargs -n1 git clone < gh-repos.txt

This clones all the repositories on the list into the current folder. This same one-liner works for GitLab as well, if you substitute the appropriate URLs.

这会将列表上的所有存储库克隆到当前文件夹中。如果替换适当的URL，则同一行也适用于GitLab。

这里发生了什么？ (What’s going on here?)

There are two halves to this one-liner: the input, counterintuitively on the right side, and the part that makes stuff happen, on the left. We could make the order of these parts more intuitive (maybe?) by writing the same command like this:

单行代码有两半：输入(在直觉上相反)在右侧，而使事情发生的部分在左侧。通过编写如下相同的命令，我们可以使这些部分的顺序更直观(也许？)：

<gh-repos.txt xargs -n1 git clone

To run a command for each line of our input, gh-repos.txt, we use xargs -n1. The tool xargs reads items from input and executes any commands it finds (it will echo if it doesn’t find any). By default, it assumes that items are separated by spaces; new lines also works and makes our list easier to read. The flag -n1tells xargs to use 1 argument, or in our case, one line, per command. We build our command with git clone, which xargs then executes for each line. Ta-da.

要对输入的每一行gh-repos.txt运行命令，我们使用xargs -n1 。 xargs工具从输入中读取项目并执行找到的所有命令(如果找不到则echo )。默认情况下，它假定项目之间用空格隔开；新行也可以使我们的列表更易于阅读。标志-n1告诉xargs每个命令使用1参数，在本例中为1行。我们使用git clone构建命令，然后xargs将针对每一行执行。 -

Bash一线式工具，可在GitLab上创建并推送许多存储库 (A Bash one-liner to create and push many repositories on GitLab)

GitLab, unlike GitHub, lets us do this nifty thing where we don’t have to use the website to make a new repository first. We can create a new GitLab repository from our terminal. The newly created repository defaults to being set as Private, so if we want to make it Public on GitLab, we’ll have to do that manually later.

与GitHub不同，GitLab让我们可以做这件漂亮的事情，而不必先使用网站创建新的存储库。我们可以从终端创建一个新的GitLab存储库。新创建的存储库默认设置为“私有”，因此，如果要在GitLab上将其设置为“公共”，则稍后必须手动进行。

The GitLab docs tell us to push to create a new project using git push --set-upstream, but I don’t find this to be very convenient for using GitLab as a backup. As I work with my repositories in the future, I’d like to run one command that pushes to both GitHub and GitLab without additional effort on my part.

GitLab文档告诉我们使用git push --set-upstream来推动创建一个新项目，但是我发现使用GitLab作为备份不是很方便。将来在使用存储库时，我想运行一个命令同时推送到GitHub 和 GitLab，而无需我付出额外的努力。

To make this Bash one-liner work, we’ll also need a list of repository URLs for GitLab (ones that don’t exist yet). We can easily do this by copying our GitHub repository list, opening it up with Vim, and doing a search-and-replace:

为了使此Bash单线工作，我们还需要GitLab的存储库URL列表(尚不存在的URL)。我们可以轻松地做到这一点，方法是复制我们的GitHub存储库列表，使用Vim打开它，然后进行搜索和替换：

cp gh-repos.txt gl-repos.txt
vim gl-repos.txt
:%s/\<github\>/gitlab/g
:wq

This produces gl-repos.txt, which looks like:

这将产生gl-repos.txt ，看起来像：

git@gitlab.com:username/first-repository.git
git@gitlab.com:username/second-repository.git
git@gitlab.com:username/third-repository.git

We can create these repositories on GitLab, add the URLs as remotes, and push our code to the new repositories by running:

我们可以在GitLab上创建这些存储库，将URL添加为远程存储，并通过运行以下命令将代码推送到新的存储库：

awk -F'\/|(\.git)' '{system("cd ~/FULL/PATH/" $2 " && git remote set-url origin --add " $0 " && git push")}' gl-repos.txt

Hang tight and I’ll explain it; for now, take note that ~/FULL/PATH/ should be the full path to the directory containing our GitHub repositories.

请稍等，我会解释。现在，请注意~/FULL/PATH/应该是包含我们的GitHub存储库的目录的完整路径。

We do have to make note of a couple assumptions:

我们必须注意以下两个假设：

The name of the directory on your local machine that contains the repository is the same as the name of the repository in the URL (this will be the case if it was cloned with the one-liner above);
包含资源库的本地计算机上目录的名称与URL中资源库的名称相同(如果是使用上述单行代码克隆的，则为这种情况)；
Each repository is currently checked out to the branch you want pushed, ie. master.
当前每个存储库都签出到要推送的分支，即。 master 。

The one-liner could be expanded to handle these assumptions, but it is the humble opinion of the author that at that point, we really ought to be writing a Bash script.

单行代码可以扩展以处理这些假设，但是作者的拙见认为，在那时，我们确实应该编写Bash脚本。

这里发生了什么？ (What’s going on here?)

Our Bash one-liner uses each line (or URL) in the gl-repos.txt file as input. With awk, it splits off the name of the directory containing the repository on our local machine, and uses these pieces of information to build our larger command. If we were to print the output of awk, we’d see:

我们的Bash单行使用gl-repos.txt文件中的每一行(或URL)作为输入。使用awk ，它拆分出包含本地计算机上存储库的目录名称，并使用这些信息来构建更大的命令。如果要print awk的输出， awk看到：

cd ~/FULL/PATH/first-repository && git remote set-url origin --add git@gitlab.com:username/first-repository.git && git push
cd ~/FULL/PATH/second-repository && git remote set-url origin --add git@gitlab.com:username/second-repository.git && git push
cd ~/FULL/PATH/third-repository && git remote set-url origin --add git@gitlab.com:username/third-repository.git && git push

Let’s look at how we build this command.

让我们看看我们如何构建此命令。

用`awk`分割字符串 (Splitting strings with `awk`)

The tool awk can split input based on field separators. The default separator is a whitespace character, but we can change this by passing the -F flag. Besides single characters, we can also use a regular expression field separator. Since our repository URLs have a set format, we can grab the repository names by asking for the substring between the slash character / and the end of the URL, .git.

工具awk可以基于字段分隔符拆分输入。默认的分隔符是空格字符，但是我们可以通过传递-F标志来更改它。除了单个字符，我们还可以使用正则表达式字段分隔符。由于我们的存储库URL具有固定的格式，因此我们可以通过请求斜杠/和URL末尾.git之间的子字符串来获取存储库名称。

One way to accomplish this is with our regex \/|(\.git):

实现此目的的一种方法是使用我们的正则表达式\/|(\.git) ：

\/ is an escaped / character;
\/是转义的/字符；
| means “or”, telling awk to match either expression;
| 表示“或”，告诉awk匹配任一表达式；
(\.git) is the capture group at the end of our URL that matches “.git”, with an escaped . character. This is a bit of a cheat, as “.git” isn’t strictly splitting anything (there’s nothing on the other side) but it’s an easy way for us to take this bit off.
(\.git)是URL末尾的捕获组，与“ .git”匹配，并带有一个转义的. 字符。这有点作弊，因为“ .git”并没有严格分割任何内容(另一面没有任何内容)，但这是我们轻松实现这一点的简便方法。

Once we’ve told awk where to split, we can grab the right substring with the field operator. We refer to our fields with a $ character, then by the field’s column number. In our example, we want the second field, $2. Here’s what all the substrings look like:

告诉awk拆分位置后，我们可以使用field运算符来获取正确的子字符串。我们用$字符引用字段，然后用字段的列号引用。在我们的示例中，我们需要第二个字段$2 。这是所有子字符串的样子：

1: git@gitlab.com:username
2: first-repository

To use the whole string, or in our case, the whole URL, we use the field operator $0. To write the command, we just substitute the field operators for the repository name and URL. Running this with print as we’re building it can help to make sure we’ve got all the spaces right.

要使用整个字符串，或者在我们的示例中，使用整个URL，我们使用字段运算符$0 。要编写命令，我们只需将字段运算符替换为存储库名称和URL。在构建它时使用print运行它可以帮助确保所有空间都正确。

awk -F'\/|(\.git)' '{print "cd ~/FULL/PATH/" $2 " && git remote set-url origin --add " $0 " && git push"}' gl-repos.txt

运行命令 (Running the command)

We build our command inside the parenthesis of system(). By using this as the output of awk, each command will run as soon as it is built and output. The system() function creates a child process that executes our command, then returns once the command is completed. In plain English, this lets us perform the Git commands on each repository, one-by-one, without breaking from our main process in which awk is doing things with our input file. Here’s our final command again, all put together.

我们在system()括号内构建命令。通过将其用作awk的输出，每条命令在生成并输出后将立即运行。 system()函数创建一个执行我们的命令的子进程，然后在命令完成后返回。用简单的英语来说，这使我们可以在每个存储库上一对一地执行Git命令，而不会破坏awk使用输入文件执行操作的主要过程。这是我们的最终命令，所有命令都放在一起了。

awk -F'\/|(\.git)' '{system("cd ~/FULL/PATH/" $2 " && git remote set-url origin --add " $0 " && git push")}' gl-repos.txt

使用我们的备份 (Using our backups)

By adding the GitLab URLs as remotes, we’ve simplified the process of pushing to both externally hosted repositories. If we run git remote -v in one of our repository directories, we’ll see:

通过将GitLab URL添加为远程站点，我们简化了推送到两个外部托管存储库的过程。如果在其中一个存储库目录中运行git remote -v ，我们将看到：

origin  git@github.com:username/first-repository.git (fetch)
origin  git@github.com:username/first-repository.git (push)
origin  git@gitlab.com:username/first-repository.git (push)

Now, simply running git push without arguments will push the current branch to both remote repositories.

现在，简单地运行不带参数的git push会将当前分支推送到两个远程存储库。

We should also note that git pull will generally only try to pull from the remote repository you originally cloned from (the URL marked (fetch) in our example above). Pulling from multiple Git repositories at the same time is possible, but complicated, and beyond the scope of this post. Here’s an explanation of pushing and pulling to multiple remotes to help get you started, if you’re curious. The Git documentation on remotes may also be helpful.

我们还应该注意， git pull通常只会尝试从最初克隆的远程存储库中(fetch)在上面的示例中标记为(fetch)的URL)。可以同时从多个Git存储库中拉出，但很复杂，超出了本文的范围。如果您有好奇心，这是推和拉到多个遥控器以帮助您入门的说明。遥控器上的Git文档也可能会有所帮助。

详细阐述Bash单线的简洁性 (To elaborate on the succinctness of Bash one-liners)

Bash one-liners, when understood, can be fun and handy shortcuts. At the very least, being aware of tools like xargs and awk can help to automate and alleviate a lot of tediousness in our work. However, there are some downsides.

如果了解Bash一线，可以说是有趣而便捷的捷径。至少，了解xargs和awk类的工具可以帮助自动化和减轻我们工作中的繁琐工作。但是，还有一些缺点。

In terms of an easy-to-understand, maintainable, and approachable tool, Bash one-liners suck. They’re usually more complicated to write than a Bash script using if or while loops, and certainly more complicated to read. It’s likely that when we write them, we’ll miss a single quote or closing parenthesis somewhere; and as I hope this post demonstrates, they can take quite a bit of explaining, too. So why use them?

就易于理解，易于维护和易于使用的工具而言，Bash一线难熬。与使用if或while循环的Bash脚本相比，编写它们通常更复杂，并且读取起来当然更复杂。当我们编写它们时，可能会在某处遗漏单引号或右括号。正如我希望这篇文章所演示的那样，他们也可以做很多解释。那为什么要使用它们呢？

Imagine reading a recipe for baking a cake, step by step. You understand the methods and ingredients, and gather your supplies. Then, as you think about it, you begin to realize that if you just throw all the ingredients at the oven in precisely the right order, a cake will instantly materialize. You try it, and it works!

想象一下，逐步阅读烘焙蛋糕的食谱。您了解方法和成分，并收集物品。然后，当您考虑它时，您开始意识到，如果只是以正确的顺序将所有配料扔进烤箱，蛋糕就会立即变质。您尝试一下，就可以了！

That would be pretty satisfying, wouldn’t it?

那会很令人满意，不是吗？