netflix

Netflix open source Polynote is a new notebook environment and was born out of the necessity to accelerate data science experimentation at Netflix.

Netflix开源Polynote是一种新的笔记本环境，其诞生是出于加速Netflix数据科学实验的需要。

Over the last few years, Netflix has transformed its use of data science notebooks from an experimentation artifact to a key component of the lifecycle of machine learning solutions. Initially, Netflix adopted Jupyter Notebooks like a data exploration and analysis tools.

在过去的几年中，Netflix已将其对数据科学笔记本的使用从实验工件转变为机器学习解决方案生命周期的关键组成部分。最初，Netflix将Jupyter Notebooks用作数据探索和分析工具。

Polynote (Polynote)

Polynote is a multi-language notebook experimentation environment. In addition to Python, the current release supports languages such as SQL, Vega(visualizations) and, of course, Scala. The platform is also integrated with data science infrastructures such as Apache Spark. At its core, Polynote includes the following capabilities:

Polynote是一种多语言笔记本实验环境。除Python外 ，当前版本还支持SQL ， Vega (可视化)，当然还有Scala等语言 。该平台还与数据科学基础架构(例如Apache Spark)集成在一起。 Polynote的核心包括以下功能：

a) Improved Editing Experience: Polynote tries to enable an editing experience closer to modern IDEs. b) Multi-Language Support: Polynote introduces first-class support for Scala and other languages used in data science environmenhts. c) Data Visualization Improvements: Polynote integrates native data visualizations into notebooks’ dataset without the need of adding a lot of code. d) Configuration and Dependency Management: Languages like Scala require complex package dependencies in its programs. Polynote saves the package dependency configuration within the notebook itself addressing some of the common challenges in this area experienced by JVM developers. e) Reproducibility: The combination of code, data and execution results into a single document makes notebooks powerful, but also difficult to reproduce. Polynote includes reproducibility as a first-class capability of the framework.

a) 改进的编辑体验： Polynote试图使编辑体验更接近现代IDE。 b) 多语言支持： Polynote引入了对Scala和数据科学环境中使用的其他语言的一流支持。 c) 数据可视化方面的改进： Polynote将原生数据可视化集成到笔记本的数据集中，而无需添加大量代码。 d) 配置和依赖性管理：诸如Scala之类的语言在其程序中需要复杂的软件包依赖性。 Polynote将包依赖项配置保存在笔记本自身中，以解决JVM开发人员在该领域遇到的一些常见挑战。 e)可复制性：将代码，数据和执行结果组合到一个文档中，使笔记本功能强大，但也难以复制。 Polynote将可再现性作为框架的一流功能。

改进的编辑体验 (Improved Editing Experience)

Polynote includes common features in IDEs such as code auto-completion or syntax error highlighting which improves the experience for data scientists and researchers building Notebooks. More of the editing capabilities are powered by the Monaco editor which powers the experience of Visual Studio Code.

Polynote包含IDE中的常见功能，例如代码自动完成或语法错误突出显示，从而改善了构建笔记本电脑的数据科学家和研究人员的体验。摩纳哥编辑器提供了更多的编辑功能，该编辑器为Visual Studio Code的体验提供了支持。

多国语言支持 (Multi-Language Support)

Polynote does not only provide support for multiple languages but it also allows those languages to be combined in a single program. In Polynote, every cell can be based on a different language. When a cell is run, the kernel provides the available typed input values to the cell’s language interpreter. In turn, the interpreter provides the resulting typed output values back to the kernel. This allows cells in Polynote notebooks to operate within the same context. The example below shows a Python library, to compute an isotonic regression of a dataset generated with Scala.

Polynote不仅提供对多种语言的支持，而且还允许将这些语言组合在一个程序中。在Polynote中，每个单元格可以基于不同的语言。当单元运行时，内核将可用的类型化输入值提供给单元的语言解释器。反过来，解释器将结果输入的输出值提供回内核。这使Polynote笔记本中的单元格可以在相同的上下文中运行。下面的示例显示了一个Python库，用于计算使用Scala生成的数据集的等渗回归。

Image for post — Source: https://polynote.org/

数据可视化 (Data Visualization)

Data visualizations are a common component of most notebook environment. However, Polynote takes the visualization value proposition to another level by including it as a native component of the platform which does not require developers to write any code in order to visually explore a dataset.

数据可视化是大多数笔记本环境的常见组件。但是，Polynote通过将可视化价值主张作为平台的本机组件，将可视化价值主张提升到了另一个层次，不需要开发人员编写任何代码即可直观地浏览数据集。

组态 (Configuration)

Most of the time, data scientists working on notebooks can enjoy the efficiency of Python’s package management model to handle the dependencies of a program. However, in JVM-languages like Scala dependency management can become a total night mare. Polynote addresses that challenge by storing the configuration and dependency information directly in the notebook itself, rather than relying on external files. Additionally, Polynote provides a user-friendly Configuration section where users can set dependencies for each notebook.

大多数时候，从事笔记本工作的数据科学家可以享受Python的包管理模型处理程序依赖关系的效率。但是，在诸如Scala依赖关系管理之类的JVM语言中，它们可能会变成一头噩梦。 Polynote通过将配置和相关性信息直接存储在笔记本本身中而不是依赖于外部文件来解决这一挑战。此外，Polynote还提供了一个用户友好的“配置”部分，用户可以在其中为每个笔记本设置依赖性。

重现性 (Reproducibility)

With Polynote, Netflix a new code interpretation block instead of relying on a REPL model like a traditional notebook. One of the key capabilities of the new interpretation model is that it removes hidden states which allows data scientists to copy cells within a notebook without introducing any state from the previous position.

借助Polynote，Netflix有了新的代码解释模块，而不再像传统笔记本那样依赖REPL模型。新解释模型的关键功能之一是，它消除了隐藏状态，这使数据科学家可以在笔记本中复制单元而无需从先前位置引入任何状态。

Polynote is a new release in the ambitious competitive of data science notebooks but one that stands in its own merits. The support for JVM-based languages could make Polynote a favorite of developers working on Spark infrastructures. Also the editing and reproducatility capabilities are definitely welcomed enhancements to traditional notebook environments. Polynote is available in Github and you can also follow the project’s website.

Polynote是在雄心勃勃的数据科学笔记本电脑竞争中推出的新版本，但它有自己的优点。对基于JVM的语言的支持可能使Polynote成为使用Spark基础结构的开发人员的最爱。同样，编辑和再现性功能无疑是对传统笔记本环境的增强。 Polynote 在Github中可用，您也可以访问该项目的网站。