By: Content by Edward Krueger and Josh Farmer, and Douglas Franklin.

作者： 爱德华·克鲁格 ( Edward Krueger) 和 乔什·法默 ( Josh Farmer )以及 道格拉斯·富兰克林 ( Douglas Franklin)的内容 。

When building an application that performs time-consuming, complex, or resource-intensive tasks, it can be frustrating to wait for these to complete within the front end application. Additionally, complex tasks in the front end can time-out. Redis Queue fixes this by pushing more sophisticated tasks to a worker for processing.

在构建执行耗时，复杂或资源密集型任务的应用程序时，等待这些任务在前端应用程序中完成可能会令人沮丧。 此外，前端的复杂任务可能会超时。 Redis Queue通过将更复杂的任务推送给工作人员进行处理来解决此问题。

Using Redis with Redis Queue allows you to enter those complex tasks into a queue, so the Redis worker executes these tasks outside of your application’s HTTP server.

通过将Redis与Redis Queue一起使用，您可以将那些复杂的任务输入队列，因此Redis worker在应用程序的HTTP服务器之外执行这些任务。

In this article, we will build an app that enqueues jobs with Redis queue, performs a function on those jobs and returns the result of the function.

在本文中，我们将构建一个应用程序，该应用程序使用Redis队列使作业排队，对这些作业执行功能，并返回该功能的结果。

Here is the link to the Github Repository with our code for this project.

这是Github信息库的链接，以及该项目的代码。

什么是Redis？ (What is Redis?)

Redis is an open-source, in-memory database, cache, and message broker. Messages handled by Redis are essentially JSONs. Redis is ideal for quickly and easily working with specific data types in a temporary database, and provides very rapid access and delivery of queries.

Redis是一个开源的内存数据库，缓存和消息代理。 Redis处理的消息本质上是JSON。 Redis是快速，轻松地使用临时数据库中特定数据类型的理想选择，并且可以非常快速地访问和交付查询。

For us, Redis offers two benefits. First, it pushes complex tasks to another space to be processed. Second, it is easier for the developer to handle complex actions by splitting the task into separate functions, the main application and the queue.

对于我们来说，Redis提供了两个好处。 首先，它将复杂的任务推到另一个要处理的空间。 其次，通过将任务分为独立的功能，主应用程序和队列，开发人员可以更轻松地处理复杂的操作。

For this application, we’ll be using Redis to hold our queue of JSON messages. Redis can be a stand-alone database, accessible by many computers or systems. In our example, we will be using it as a local memory store to support our application.

对于此应用程序，我们将使用Redis来保存JSON消息队列。 Redis可以是一个独立的数据库，可以被许多计算机或系统访问。 在我们的示例中，我们将使用它作为本地内存存储来支持我们的应用程序。

什么是Redis Queue？ (What is Redis Queue?)

Redis Queue is a python library for queueing jobs for background processing. Since many hosting services will time out on long HTTP requests, it is best to design APIs to close requests as quickly as possible. Redis Queue allows us to do this by pushing tasks to a queue and then to a worker for processing.

Redis Queue是一个用于对作业进行排队以进行后台处理的python库。 由于许多托管服务会在较长的HTTP请求上超时，因此最好设计API以尽快关闭请求。 Redis Queue允许我们通过将任务推入队列，然后推给工作人员进行处理来做到这一点。

Using Redis in conjunction with Redis Queue allows you to request input from the user, return a validation response to the user, and queue up processes in the background. All without the front end user having to wait for those processes to complete. Processes could be anything from Machine Learning models, to duplicating an image to complex simulations.

将Redis与Redis Queue结合使用可以使您从用户请求输入，向用户返回验证响应，并在后台排队进程。 所有这些都无需前端用户等待这些过程完成。 从机器学习模型到将图像复制到复杂的模拟，过程可以是任何东西。

Anything that takes longer to complete than you would like to have taken place on the front end of your application belongs in the Redis Queue.

Redis队列中需要完成的时间比在应用程序前端所需的时间长。

模块化应用程序结构 (Modular App Structure)

We will cover using Docker to run the Redis database and initialize the Redis Queue worker. The worker will allow us to process the jobs in our application’s queue.

我们将介绍如何使用Docker运行Redis数据库并初始化Redis Queue worker。 工作者将允许我们处理应用程序队列中的作业。

We’ll go over each of these files in more detail in the following sections.

在以下各节中，我们将详细介绍每个文件。

We’ll discuss the following constituent files of our app, main.py, functions.py, and redis_resc.py.

我们将讨论应用程序的以下组成文件： main.py ， functions.py和redis_resc.py 。

There are two ways to initiate the docker container. We will be covering the standard process one step at a time in this article. However, the docker container can be created using docker-compose, which we will cover in a separate article.

有两种启动docker容器的方法。 本文将一次一步地介绍标准过程。 但是，可以使用docker-compose创建docker容器，我们将在另一篇文章中介绍。

安装Docker和Redis (Installing Docker and Redis)

Docker is a containerization service. This means Docker runs code within a container that has within it the dependencies required for the application to run reliably from one system to another. It ensures our applications run consistently, from development to testing to production.

Docker是一种容器化服务。 这意味着Docker在容器中运行代码，该容器中包含应用程序从一个系统可靠地运行到另一个系统所需的依赖关系。 它确保我们的应用程序从开发到测试再到生产，始终如一地运行。

Follow this link to the Docker documentation for a comprehensive installation guide to help you select the correct version of Docker for your operating system.

单击此链接以获取完整的安装指南的Docker文档 ，以帮助您为您的操作系统选择正确的Docker版本。

Once you have Docker installed on your system, you will need to install Redis.

一旦在系统上安装了Docker，就需要安装Redis。

For Mac users, the home-brew command brew install redis will work. Windows requires a download from Github. With Linux, you can download the latest tarball directly from Redis.

对于Mac用户，home-brew命令brew install redis将起作用。 Windows需要从Github下载 。 使用Linux，您可以直接从Redis下载最新的tarball。

自述概述 (Readme Overview)

To run our application, we must understand the instructions on the Readme file.

要运行我们的应用程序，我们必须了解自述文件上的说明 。

We will be using Pipenv to create a virtual environment and install the appropriate packages. Install the packages for this program with the command pipenv install -- dev. Adding the --dev flag installs development packages and production requirements. Before moving to the next step, activate the environment with the command pipenv shell.

我们将使用Pipenv创建虚拟环境并安装适当的软件包。 使用命令pipenv install -- dev该程序的软件包。 添加--dev标志将安装开发包和生产要求。 进行下一步之前，请使用命令pipenv shell激活环境。

We will have to start three services to have the app function correctly: Redis Database, a Redis Queue worker, and the Flask application. Two of these services will begin within the same terminal as the one we used to install our virtual environment and enter the shell.

我们将必须启动三个服务才能使应用程序正常运行：Redis数据库，Redis队列工作器和Flask应用程序。 这些服务中的两项将在与我们用于安装虚拟环境并进入Shell的终端相同的终端中启动。

First, run the command:

首先，运行命令：

docker pull redis

This will pull the Redis image from Docker Hub.

这将从Docker Hub中提取Redis映像。

Then, run the command:

然后，运行命令：

docker run -d -p 6379:6379 redis

This command serves several purposes. First, it initializes the docker container. The -d flag runs the Redis container in the background, freeing up your terminal for the next step. The -p flag publishes your container’s port or ports to the host. Lastly, 6379:6379 binds port 6379 inside the docker container to port 6379 on the localhost. Without binding the port, you will run into issues when the local machine and processes within your docker container attempt to communicate with one another.

此命令有几个用途。 首先，它初始化docker容器。 -d标志在后台运行Redis容器，从而释放您的终端以进行下一步。 -p标志将容器的一个或多个端口发布到主机。 最后， 6379:6379将docker容器内的端口6379:6379绑定到本地主机上的端口6379。 如果不绑定端口，当docker容器中的本地计算机和进程尝试相互通信时，就会遇到问题。

The final step before running our application is starting the Redis Queue worker. Since we are running the Redis image in detached mode, we can do this step within the same terminal.

运行我们的应用程序之前的最后一步是启动Redis Queue worker。 由于我们以分离模式运行Redis映像，因此我们可以在同一终端中执行此步骤。

Start the Redis worker with the command rq worker . The rq command is only available to the terminal if the Redis Queue package is installed, we installed the dependencies with Pipenv install, and entered the environment with pipenv shell. Running the rq command will allow Redis Queue to begin listening for jobs to enter into the queue and start processing these jobs.

使用命令rq worker启动Redis rq worker 。 仅当安装了Redis Queue软件包，使用Pipenv install安装依赖项并使用pipenv shell进入环境时， rq命令才对终端可用。 运行rq命令将使Redis Queue开始侦听要进入队列的作业并开始处理这些作业。

初始化Flask应用 (Initializing the Flask App)

We are now ready to initialize the flask app. On a development server, this is done with the command:

现在我们准备初始化flask应用程序。 在开发服务器上，这是通过以下命令完成的：

export FLASK_APP=app.main:app && flask run — reload

The command gunicorn app.main:app will run this on a production server. Gunicorn requires a Unix platform to run. Therefore, this command will work if you are using macOS or a Linux based system, but not for windows.

命令gunicorn app.main:app将在生产服务器上运行此命令。 Gunicorn需要Unix平台才能运行。 因此，如果您使用的是macOS或基于Linux的系统，则此命令将起作用，但不适用于Windows。

Now that the services are running, let’s move to the app files.

现在，这些服务正在运行，让我们移至应用程序文件。

Flask应用程序组件 (Flask App Components)

redis_resc.py (redis_resc.py)

This file sets up the Redis connection and the Redis Queue.

此文件设置Redis连接和Redis队列。

"""Sets up the redis connection and the redis queue."""
import osimport redis
from rq import Queueredis_conn = redis.Redis(host=os.getenv("REDIS_HOST", "127.0.0.1"),port=os.getenv("REDIS_PORT", "6379"),password=os.getenv("REDIS_PASSWORD", ""),
)redis_queue = Queue(connection=redis_conn)

The variable redis_conn defines the connection parameters for this connection, and redis_queue uses the Queue method. The Queue method initiates the queue, and any name can be given. Common naming patters are ‘low,’ ‘medium’ and ‘high.’ By providing the Queue method no args, we are instructing it to use the default queue. Aside from the name, we are simply passing our method the connection string to the Redis store.

变量redis_conn定义了此连接的连接参数，并且redis_queue使用Queue方法。 Queue方法启动队列，并且可以指定任何名称。 常见的命名方式是“低”，“中”和“高”。 通过不为Queue方法提供args，我们指示它使用默认队列。 除了名称之外，我们只是将连接字符串传递给Redis存储。

functions.py (functions.py)

Functions.py is where we define the functions to use in the Redis queue.

Functions.py是我们定义要在Redis队列中使用的函数的位置。

What some_long_function does is up to you. It can be anything you want with done with the data passed to the worker. This could be anything from machine learning, to image manipulation, to using the information to process queries in a connected database. Anything you want the backend to do so the front end doesn't wait would be placed within this function.

some_long_function功能取决于您。 完成传递给工作程序的数据后，您可以执行任何操作。 从机器学习到图像处理，再到使用信息来处理连接的数据库中的查询，都可以是任何东西。 您希望后端执行任何使前端不等待的操作都将放置在此函数中。

For our example, this function serves a couple of purposes. First, the job variable is created to retrieve the current job being worked. We use time.sleep(10) to demonstrate the different job status codes as the job is being processed. We will go into more detail on the job status codes later in the article. Finally, we return a JSON that contains information about the job and the results of processing the job.

在我们的示例中，此功能有两个目的。 首先，创建作业变量以检索当前正在工作的作业。 在处理作业时，我们使用time.sleep(10)演示不同的作业状态代码。 我们将在本文后面详细介绍作业状态代码。 最后，我们返回一个JSON，其中包含有关作业和处理作业的结果的信息。

主程序 (Main.py)

Main.py is the primary flask app that drives the application. More details on setting up a flask app are outlined in this medium article.

Main.py是驱动该应用程序的主要烧瓶应用程序。 这篇中篇文章概述了有关设置flask应用程序的更多详细信息。

The first route function, resource_not_found , is designed to handle any 404 errors that result from incorrect requests to the application. The route function defined as home is the default route that runs when you navigate to the home route. This route runs when you perform a GET request to http://127.0.0.1:8000 or the localhost URL. This route designed to let you know the flask app is running.

第一个路由函数resource_not_found旨在处理由于对应用程序的不正确请求而导致的任何404错误。 定义为home的路由功能是导航到本地路由时运行的默认路由。 当您执行对http://127.0.0.1:8000或本地主机URL的GET请求时，此路由将运行。 此路由旨在让您知道flask应用程序正在运行。

We can test it with Postman as seen below.

我们可以使用邮递员对其进行测试，如下所示。

The remaining routes, specific for the Redis Queue, will be covered in more detail below.

其余的路由(特定于Redis队列)将在下面详细介绍。

/入队 (/enqueue)

The /enqueue route creates the task and enters the task into the queue. Redis Queue accepts things such as key-value pairs so that you can create a post request with a dictionary, for example:

/enqueue路由创建任务并将任务输入队列。 Redis Queue接受键值对之类的东西，以便您可以使用字典创建发布请求，例如：

{“hello”: “world”}

Submitting a JSON post request to the /enqueue route will tell the application to enqueue the JSON to the function some_long_function within the function.py file for processing.

向/enqueue路由提交JSON发布请求将告诉应用程序将JSON排队到function.py文件中的some_long_function 函数进行处理。

The /enqueue route returns the job_id of the task created once it has done so. Take note of the job_id, as we will use it as part of the URL string for the application’s remaining routes.

/enqueue路由返回创建的任务的job_id 。 请注意job_id ，因为我们会将其用作应用程序其余路由的URL字符串的一部分。

/检查状态 (/check_status)

This route takes the job_id and checks its status in the Redis Queue. The full URL of this route is http://127.0.0.1:8000?job_id=JobID. Where JobID is the job_id that was generated when the /enqueue route created the job.

该路由采用job_id并在Redis队列中检查其状态。 该路由的完整URL为http://127.0.0.1:8000?job_id=JobID 。 其中JobID是/enqueue路由创建作业时生成的job_id 。

The /check_status route will return a JSON that contains the job_id passed in the URL and the status of the job using the get_status method of rq.job. Remember within the some_long_function, the time.sleep at the start of the function. If you hit this route with a get request before the ten-second timer, the status will show ‘Queue.’ After some_long_function completes, the status will return as ‘finished.’

/check_status路由将返回一个JSON，其中包含使用URL中传递的job_id和使用get_status方法的工作状态。 还记得内some_long_function的time.sleep在函数的开始。 如果您在十秒计时器之前通过获取请求触及此路线，则状态将显示“队列”。 some_long_function完成后，状态将返回“完成”。

/ get_result (/get_result)

Once a job is shown as ‘finished’ by the /check_status route, get_result will return the results of the processed job. The results are determined by what you have set up within some_long_function. This URL follows the same structure as /check_status so it is:

一旦通过/check_status route将作业显示为“完成”， get_result将返回已处理作业的结果。 结果取决于您在some_long_function设置的some_long_function 。 该URL与/check_status结构相同，因此为：

http://127.0.0.1:8000?get_result=JobID

Our some_long_function returns data about the job, as seen in the image above.

我们的some_long_function返回有关作业的数据，如上图所示。

结论 (Conclusion)

In this article, we learned a little about building a queue of tasks with Redis Queue, and how to utilize a Redis database within a docker container. We covered how to build the application that makes it simple to enqueue time-consuming tasks without tying up or hindering our front end performance.

在本文中，我们了解了一些有关使用Redis Queue构建任务队列的知识，以及如何在Docker容器中利用Redis数据库。 我们介绍了如何构建应用程序，以使其轻松地完成耗时的任务，而又不会增加或妨碍我们的前端性能。

Here is the link to the Github Repository with our code for this project. Test it out for yourself and try queueing a few JSON tasks and viewing the results!

这是Github信息库的链接，以及该项目的代码。 自己进行测试，然后尝试排队一些JSON任务并查看结果！