webgl 着色器

by Dan Ruta

通过Dan Ruta

在WebAssembly中使用WebGL着色器 (Using WebGL shaders in WebAssembly)

WebAssembly is blazing fast for number crunching, game engines, and many other things, but nothing can quite compare to the extreme parallelization of shaders, running on the GPU.

WebAssembly正在为数字运算，游戏引擎和许多其他事情快速发展，但是没有什么可以与在GPU上运行的着色器的极端并行化相比。

This is especially so if you’re looking to do some image processing. Usually, on the web, this is done through WebGL, but how would you access its APIs when using WebAssembly?

如果您要进行一些图像处理，则尤其如此。通常，在Web上，这是通过WebGL完成的，但是使用WebAssembly时如何访问其API？

配置 (Setting up)

We’ll very briefly go through setting up an example project, then we’ll look at how an image can be loaded as a texture. Then, in a separate context, we’ll apply an edge detection GLSL shader to the image.

我们将简要地设置一个示例项目，然后看一下如何将图像作为纹理加载。然后，在单独的上下文中，我们将对图像应用边缘检测GLSL着色器。

All the code is in a repo here, if you’d prefer to jump straight to that. Note that you have to serve your files via a server for WebAssembly to work.

如果您希望直接跳转到此处，所有代码都在此处的存储库中。请注意，您必须通过服务器提供文件才能使WebAssembly正常工作。

As a prerequisite, I’m going to assume you already have your WebAssembly project set up. If not, you can check out the article here on how to do it, or just fork the repo linked above.

作为前提，我将假设您已经设置了WebAssembly项目。如果没有，您可以在此处查看有关操作方法的文章，也可以只分叉上面链接的仓库。

For demoing the below code, I’m using a basic html file which serves only to load an image, get its imageData, and pass it to the WebAssembly code using the ccallArrays function.

为了演示以下代码，我使用一个基本的html文件，该文件仅用于加载图像，获取其imageData并使用ccallArrays函数将其传递给WebAssembly代码。

As for the C++ code, there is an emscripten.cpp file which manages and routes method calls to context instances created in the Context.cpp file. The Context.cpp file is structured as follows:

对于C ++代码，有一个emscripten.cpp文件，用于管理方法调用并将其路由到Context.cpp文件中创建的上下文实例。 Context.cpp文件的结构如下：

汇编 (Compilation)

WebGL is based on and follows the OpenGL ES (Embedded Systems) spec, which is a subset of OpenGL. When compiling, emscripten will map our code to the WebGL API.

WebGL基于并遵循OpenGL ES(嵌入式系统)规范，该规范是OpenGL的子集。编译时，emscripten会将我们的代码映射到WebGL API。

There are a couple of different versions we can target. OpenGL ES 2 maps to WebGL 1, whereas OpenGL ES 3 maps to WebGL 2. By default you should target WebGL 2, as it comes with some free optimizations and improvements.

我们可以定位几个不同的版本。 OpenGL ES 2映射到WebGL 1，而OpenGL ES 3映射到WebGL2。默认情况下，您应该以WebGL 2为目标，因为它带有一些免费的优化和改进功能。

To do this, we must add the USE_WEBGL2=1 flag to the compilation.

为此，我们必须在编译中添加USE_WEBGL2=1标志。

If you are planning to use some OpenGL ES features not present in the WebGL spec, you can use the FULL_ES2=1 and/or FULL_ES3=1 flags.

如果您打算使用WebGL规范中未提供的某些OpenGL ES功能，则可以使用FULL_ES2=1和/或FULL_ES3=1标志。

To be able to handle large textures/images, we can also add the ALLLOW_MEMORY_GROWTH=1 flag. This removes the memory limit of the WebAssembly program, at the cost of some optimizations.

为了能够处理大的纹理/图像，我们还可以添加ALLLOW_MEMORY_GROWTH=1标志。这以一些优化为代价，消除了WebAssembly程序的内存限制。

If you know ahead of time how much memory you’ll need, you can instead use the TOTAL_MEMORY=X flag, where X is the memory size.

如果您提前知道需要多少内存，则可以使用TOTAL_MEMORY=X标志，其中X是内存大小。

So we’re going to end up with something like this:

因此，我们将最终得到如下结果：

emcc -o ./dist/appWASM.js ./dev/cpp/emscripten.cpp -O3 -s ALLOW_MEMORY_GROWTH=1 -s USE_WEBGL2=1 -s FULL_ES3=1 -s WASM=1 -s NO_EXIT_RUNTIME=1 -std=c++1z

Finally, we need the following imports, in our code:

最后，我们需要在代码中进行以下导入：

#include <emscripten.h>#include <string>#include <GLES2/gl2.h>#include <EGL/egl.h>extern "C" {       #include "html5.h" // emscripten module}

实作 (Implementation)

If you have previous experience with WebGL or OpenGL, then this bit may seem familiar.

如果您以前有使用WebGL或OpenGL的经验，那么这一点可能看起来很熟悉。

When writing OpenGL, the API will not work until you create a context. This is normally done using platform specific APIs. However, the web is not platform bound, and we can instead use an API integrated into OpenGL ES.

在编写OpenGL时，除非创建上下文，否则该API将无法工作。通常使用平台特定的API完成此操作。但是，网络不受平台限制，因此我们可以使用集成到OpenGL ES中的API。

The majority of the legwork, however, can be more easily implemented using emscripten’s APIs in the html5.h file. The functions we’re interested in are:

但是，使用html5.h文件中的emscripten的API可以更轻松地实现大部分工作。我们感兴趣的功能是：

emscripten_webgl_create_context — This will instantiate a context for the given canvas and attributes
emscripten_webgl_create_context —这将为给定的画布和属性实例化上下文
emscripten_webgl_destroy_context — This is needed for cleaning up memory when destructing context instances
emscripten_webgl_destroy_context —销毁上下文实例时清理内存是必需的
emscripten_webgl_make_context_current — This will assign and switch which context WebGL will render to
emscripten_webgl_make_context_current —这将分配并切换WebGL渲染到的上下文

创建上下文 (Create the context)

To start implementing, you have to first create the canvas elements in your JavaScript code. Then, when using the emscripten_webgl_create_context function, you pass the id of the canvas as the first parameter, with any configurations as the second. The emscripten_webgl_make_context_current function is used to set the new context as the one currently in use.

要开始实施，您必须首先在JavaScript代码中创建canvas元素。然后，在使用emscripten_webgl_create_context函数时，您将画布的ID作为第一个参数传递，将任何配置作为第二个参数传递。 emscripten_webgl_make_context_current函数用于将新上下文设置为当前使用的上下文。

Next, the vertex shader (to specify coordinates) and the fragment shader (to calculate the colour at each pixel) are both compiled, and the program is built.

接下来，编译顶点着色器(用于指定坐标)和片段着色器(以计算每个像素的颜色)，并构建程序。

Finally, the shaders are attached to the program, which is then linked, and validated.

最后，将着色器附加到程序，然后将其链接并验证。

Though that sounds like a lot, the code for this is as follows:

尽管这听起来很多，但其代码如下：

The shader compilation is done within the CompileShader helper function which performs the compilation, printing out any errors:

着色器编译是在CompileShader帮助器函数中完成的，该函数执行编译，并打印出所有错误：

创建着色器 (Create the shader)

The shader code for this example is minimal, and it just maps each pixel to itself, to display the image as a texture:

此示例的着色器代码很少，它仅将每个像素映射到自身，以将图像显示为纹理：

You can access the canvas’ context in JavaScript in addition to the context in the C++ code, but it must be of the same type, ‘webgl2’. While defining multiple context types does nothing when just using JavaScript, if you do it before creating the webgl2 context in WebAssembly, it will throw an error when the code execution gets there.

除了C ++代码中的上下文，您还可以在JavaScript中访问canvas的上下文，但是它必须是相同的类型“ webgl2”。尽管仅使用JavaScript定义多个上下文类型并没有任何作用，但是如果在WebAssembly中创建webgl2上下文之前进行了定义，则在执行代码时会引发错误。

加载纹理 (Loading the texture)

The first thing to do when applying the shader is to call the emscripten_webgl_make_context_currentfunction to make sure that we are still using the correct context, and glUseProgramto make sure we are using the correct program.

应用着色器时要做的第一件事是调用emscripten_webgl_make_context_current函数以确保我们仍在使用正确的上下文，并glUseProgram来确保我们在使用正确的程序。

Next, we get the indices of the GLSL variables (similar to getting a pointer) via theglGetAttribLocationand glGetUniformLocation functions, so we can assign our own values to those locations. The function used to do that depends on the value type.

接下来，我们通过glGetAttribLocation和glGetUniformLocation获取GLSL变量的索引(类似于获取指针) 函数，因此我们可以将自己的值分配给这些位置。用于执行此操作的函数取决于值类型。

For example, an integer, such as the texture location needs glUniform1i, whereas a float would need glUniform1f. This is a good resource for seeing which function you need to use.

例如，整数(例如纹理位置)需要glUniform1i ，而浮点数则需要glUniform1f 。这是查看需要使用哪个功能的好资源。

Next, we get the texture object via glGenTextures, assign it as the active texture, and load the imageData buffer. The vertex and indices buffers are then bound, to set the boundaries of the texture to fill the canvas.

接下来，我们通过glGenTextures获得纹理对象，将其分配为活动纹理，并加载imageData缓冲区。然后绑定顶点和索引缓冲区，以设置纹理的边界以填充画布。

Finally, we clear the existing content, define our remaining variables with data, and draw to the canvas.

最后，我们清除现有内容，使用数据定义剩余的变量，然后绘制到画布上。

使用着色器检测边缘 (Detect edges using a shader)

To add another context, where the edge detection is done, we load a different fragment shader (which applies the Sobel filter), and we bind the width and height as extra variables, in the code.

为了添加另一个完成边缘检测的上下文，我们在代码中加载了一个不同的片段着色器(应用了Sobel滤波器)，并将宽度和高度绑定为额外的变量。

To pick between different fragment shaders, for the different contexts, we just add an if-else statement in the constructor, like so:

为了在不同的片段着色器之间进行选择，针对不同的上下文，我们只需在构造函数中添加if-else语句，如下所示：

And to load the width and height variables, we add the following to the run function:

为了加载width和height变量，我们将以下内容添加到run函数中：

If you run into an error similar toERROR: GL_INVALID_OPERATION : glUniform1i: wrong uniform function for type, then there’s a mismatched assignment function for the given variable.

如果遇到类似于ERROR: GL_INVALID_OPERATION : glUniform1i: wrong uniform function for type ，则给定变量的赋值函数不匹配。

One thing to look out for when sending the imageData, is to use the correct heap, unsigned integer (the Uint8Array typed array). You can learn more about those here, but if you’re using the ccallArray function, set the ‘heapIn’ config to “HEAPU8”, as seen above.

发送imageData时要注意的一件事是使用正确的堆(无符号整数)(Uint8Array类型的数组)。您可以在此处了解更多信息，但是，如果您使用的是ccallArray函数，则将“ heapIn ”配置设置为“ HEAPU8 ”，如上所示。

If the type is not correct, the texture will still load, but you’re going to be seeing strange renderings, like these:

如果类型不正确，纹理仍会加载，但是您将看到奇怪的渲染，如下所示：

结论 (Conclusion)

We’ve gone through a mini “Hello World!”-style project to show how to load textures and apply GLSL shaders to them in WebAssembly. The complete code is hosted on GitHub here, for further reference.

我们已经完成了一个迷你的“ Hello World！”风格的项目，以展示如何加载纹理并将GLSL着色器应用到WebAssembly中。完整的代码是在GitHub托管在这里，以备将来参考。

For a real project, you may want to add some additional error handling. I omitted it here, for clarity.

对于真实项目，您可能需要添加一些其他错误处理。为了清楚起见，我在这里省略了它。

It may also be more efficient (in the above example) to share data such as the imageData texture between contexts. You can read more about this and more here.

(在上述示例中)在上下文之间共享数据(例如imageData纹理)也可能更为有效。您可以在此处有关此内容的信息。

For some further reading, you can check out this link for common mistakes, or you can look through some demo projects in emscripten’s glbook folder, on GitHub.

若要进一步阅读，可以查看此链接以查找常见错误，也可以在GitHub上emscripten的glbook文件夹中浏览一些演示项目。

To see WebGL being used in a WebAssembly project, you can check out the dev branch on jsNet, a web based deep learning framework, where I’ll be working on moving heavier computations onto shaders, over the next few weeks (support for WebGL compute shaders via OpenGL ES 3.1 can’t come soon enough ? ).

要查看WebGL在WebAssembly项目中的使用情况，您可以在jsNet上检查一下dev分支，jsNet是一个基于Web的深度学习框架，在接下来的几周中，我将在其中致力于将较重的计算转移到着色器上(支持WebGL计算通过OpenGL ES 3.1创建的着色器还不够快吗？)。

Update

更新资料

To see what GPU compute using shaders would look like in WebAssembly, you can check out the repo for GPGPU, a small library I’m working on, with both JavaScript and WebAssembly versions.

若要查看在WebAssembly中使用着色器进行GPU计算的外观，您可以检出GPGPU (我正在使用的一个小型库，包含JavaScript和WebAssembly版本)的存储库。