Given our last blog post of the series, which can be found here :

鉴于我们在该系列中的最后一篇博客文章，可以在这里找到：

We thought it would be helpful to explain how we implemented all of the above into an on-ground experiment. We mentioned above about how the lack of a logical time-based control group forced us to pivot to geo-temporal control formation. I would like to take this opportunity to talk about an experiment we ran as part of the Dispatch team @ Rapido.

我们认为将上述所有内容如何实施到地面实验中会有所帮助。 上面我们提到了缺乏基于时间的逻辑控制组如何迫使我们转向地时控制结构。 我想借此机会谈论作为Dispatch团队@ Rapido的一部分进行的一项实验。

什么是乘车调度？ (What is a Ride Dispatch?)

The system that decides which order request (when you tap the Request Rapido button, aka the Book my Ride button, on your app) should be sent to which particular Captain(s) to ensure that the Captain reaches the customer in the quickest and most efficient way possible, is called ‘Dispatch’. It is an homage to the days of old when Taxi services were run over the telephone and a Customer who had called in for a pickup would be patched through to an Agent who would find a willing cabbie (often after multiple calls) and that driver was “dispatched” for that order.

决定哪个订单请求(当您点击应用上的Request Rapido按钮， 也就是“预订我的乘车”按钮时 )的系统应该发送给哪个特定船长，以确保船长以最快，最快捷的方式到达客户高效的方法称为“调度”。 这是对过去的日子的敬意，当时出租车服务是通过电话运行的，而要求接机的客户会被派遣到一个代理商，该代理商会找到愿意的出租车司机(通常是在多次打电话之后)，而那个司机是“派遣”该订单。

Dispatch is one of the key levers of a ride-hailing marketplace. It is one of those systems that EVERY ride request has to propagate through, hence the room for error is low, with the stakes being very high.

调度是乘车市场的关键杠杆之一。 它是每个乘车请求都必须传播的系统之一，因此错误空间很小，风险很高。

One of the first questions we had to answer while even thinking of a product to build was, “What metrics do we look at to see if marketplace conditions are being improved”? Is the ETA the gold metric for this system, or do we look at other things like Matching Time, Distance Driven by the captain to get to the customer, and cancellations from both the demand and supply sides? We definitely had to be cognizant of these metrics while evaluating any changes to our system.

我们甚至在考虑要生产的产品时，必须回答的第一个问题是：“ 我们看什么指标才能确定市场条件是否正在改善 ”？ ETA是该系统的黄金指标，还是我们要考虑其他方面，例如比赛时间，由船长驾驶到达客户的距离以及需求方和供应方的取消？ 在评估我们系统的任何更改时，我们绝对必须意识到这些指标。

在我们开始重建它之前，Dispatch @ Rapido是什么样的？ (What was Dispatch @ Rapido like before we started rebuilding it?)

Going into the rebuilding process, the current dispatch system was a simple radial system, where a customer requests a ride on the app, and the system draws a circle of radius say 2 km, and looks at all the captains in that area, calculates the crow-flying distance to the customer, and propagates the ping in order.

进入重建过程，当前的调度系统是一个简单的放射状系统，客户请求在应用程序上乘车，该系统绘制一个半径为2 km的圆，并查看该区域中的所有机长，乌鸦飞到客户的距离，并按顺序传播ping。

As a first solution, this is fine, but discerning data enthusiasts can probably find many issues with this system — how to design the optimal radius, what happens if there is a huge divider like a ring-road or a railway crossing that results in a short euclidean distance but long route based distance. In the latter case, this would be categorized as a sub-optimal match, as now the captain has to spend more time driving empty kilometers to reach the customer, and the customer gets frustrated about being matched to a captain who looks close by but takes twice the time to reach the pickup location.

作为第一个解决方案，这很好，但是有眼光的数据爱好者可能会发现此系统存在许多问题-如何设计最佳半径，如果存在像环形公路或铁路交叉路口这样的巨大分隔线而导致行驶速度变慢，会发生什么情况？欧几里得距离短，但基于路径的距离长。 在后一种情况下，这将归类为次优比赛 ，因为现在船长不得不花更多时间驾驶空旷的里程才能到达客户，并且客户对与看上去很近但是却要走近的船长感到沮丧到达取件地点的时间两次。

This specific use case can be reduced to a higher-level question: for a given pickup location, is there a corresponding nearby area that should be geo-fenced when considering it to be a part of the “dispatch radius”?

可以将这个特定的用例简化为一个更高层次的问题：对于给定的取货地点，在将其视为“派发半径”的一部分时，是否应该对相应的附近区域进行地理围栏？

Furthermore, is there a location that is potentially further away in a euclidean sense, but closer by in terms of driving time?

此外，是否存在一个可能在欧几里得距离更远但在行车时间更近的位置？

通过支付maps API可以缓解这个问题吗？ (Won’t this problem be alleviated by paying for a maps API?)

Too expensive at a per-request level. Right now, even though we are at 20% of our pre-COVID levels (and recovering every week!), servicing each request via google-maps API would be prohibitively expensive for a growing startup like Rapido, especially in these times where innovation is warranted. The goal was to deploy a smart solution, without breaking the bank, that would still have a high impact on the ground.

在每个请求级别上太贵了。 现在，即使我们的使用率达到了COVID认证前的水平的20％(并且每周都在恢复！)，对于像Rapido这样的新兴创业公司而言，通过google-maps API服务每个请求的费用实在是太高了，尤其是在这些创新的时代保证。 我们的目标是在不中断资金的情况下部署智能解决方案，这仍然会对现场产生重大影响。

建立行车时间估算 (Building the driving time estimates)

The most crucial component of a smart Dispatch system is having reliable driving time estimates. This is essentially built by leveraging the huge store of data available to us from our historical rides. As part of our internal logging, we record the time taken from :

智能调度系统最关键的部分是可靠的行驶时间估算。 这本质上是通过利用我们过去的经验为我们提供的大量数据来构建的。 作为内部日志记录的一部分，我们记录以下时间：

1. The captain to the customer aka the ETA

   客户的船长又称ETA

2. The customer’s pickup to the customer’s drop aka the Ridetime

   顾客接送顾客的乘车时间

Each part of this gives us more coverage within a city in terms of pickup-to-drop driving times. The ETA gives us short-distance coverage, and the Ridetime gives us longer-distance coverage. We combine the two sources of data and group-by at a time-of-day and a day-of-week level, remove outliers, add a few filters for the minimum amount of rides being done in that bucket to be considered valid, and store the output in a dataset to be consumed by any concerned team.

从接送车的时间来看，每个部分都为我们提供了更多城市覆盖范围。 ETA给我们短距离覆盖，而Ridetime给我们长距离覆盖。 我们将两种数据来源结合在一起，并按一天 中的某天和一周中的某天进行分组，删除异常值，添加一些过滤器以使在该存储分区中执行的最少乘车次数被视为有效，并将输出存储在数据集中以供任何相关团队使用。

设计实验 (Designing the experiment)

Once we have a pickup-to-drop driving time map, at a time-of-day and day-of-week level, we now get to the dirty work of actually designing an experiment. The first step was to answer the question of, “for a pickup location, can we find a close-by area that has a worse driving time to the source than a further away area”. I will use this segue to introduce some of the terminologies we use in this regard :

一旦有了一天中和一周中某天的上下班驾驶时间图，我们就可以开始实际设计实验的工作了。 第一步是回答以下问题：“对于接送地点，我们能找到距离源头行驶时间比远离区域更差的附近区域”的问题。 我将使用这种方法来介绍我们在这方面使用的一些术语：

source_hex : the Uber h3 derived hex8 in which the ride request originates

source_hex ：Uber h3派生的hex8，乘车请求起源于此

bad_hex : the Uber h3 derived hex8, which is closer to the source_hex geometrically, but not while driving

bad_hex ：Uber h3派生的hex8，在几何上更接近source_hex，但在行驶时不是

good_hex : the Uber h3 derived hex8, which is further away from the source_hex geometrically, but has a faster driving time than the bad_hex

good_hex ：Uber h3派生的hex8，在几何上距离source_hex较远，但是驱动时间比bad_hex快

We do this analysis at a time_of_day and day_of_week level, so a trio of HexA HexB and HexC could be mapped as : Source_hex -> HexA, Bad_hex -> HexB, Good_hex -> HexC on a Monday morning, but on a Sunday evening, it is not necessary that HexB and HexC’s relative driving times to HexA are the same. We were cognizant to not make too many dangerous assumptions here.

我们在time_of_day和day_of_week级别进行此分析，因此可以将HexA HexB和HexC的三个映射为： Source_hex-> HexA，Bad_hex-> HexB，Good_hex-> HexC在星期一的早晨，但是在周日的晚上， HexB和HexC相对于HexA的相对行驶时间不必相同。 我们意识到在这里不要做太多危险的假设。

Here the brown hex is the source_hex, the yellow hex is the bad-level-1-hex and the orange one is the good-level-2-hex. Now, from this map, it is not clear what is the reason for the increase in ridetime from yellow to brown as opposed to orange to brown. But when we look at the google maps view it becomes evident :

这里棕色的十六进制是source_hex，黄色的十六进制是坏1进制，而橙色的是好2进制。 现在，从这张地图上，不清楚从黄色到棕色而不是橙色到棕色的行驶时间增加的原因是什么。 但是，当我们查看谷歌地图视图时，它变得显而易见：

We see that the brown and orange hex8s are bifurcated by a huge railway track ( Vijayawada is one of the biggest railway junctions in the country and regularly reports trains crossing road tracks ). On the other hand, the orange hex8 has clear unfettered access to the source_hex.

我们看到棕色和橙色的hex8s被一条巨大的铁路轨道分叉(维贾亚瓦达(Vijayawada)是该国最大的铁路枢纽之一，并定期报告火车穿越道路)。 另一方面，橙色的hex8可以不受限制地访问source_hex。

Once we have the universe of such hex trios, we are back to the problem or how to do a test-control split. Given that time-based control is not an option, we tried to use features ( relevant to dispatch) of each hex-trio and passed it through a vector-similarity measure to calculate the similarity scores of each pair restricted by both of them having the same day and time at which the hex-trio is valid ( aka both test and control source hexes have bad and good hexes on the same day and time period ).

一旦有了这样的十六进制三重奏的宇宙，我们就回到了问题或如何进行测试控制拆分。 鉴于基于时间的控制不是一种选择，我们尝试使用每个十六进制三元组的功能(与分派有关)，并将其通过向量相似度度量传递，以计算受其限制的每对相似度得分。十六进制三重奏有效的同一天和同一时间(又称测试和对照源十六进制在同一天和同一时间段内有坏和好十六进制)。

It doesn’t make a lot of sense to say HexA on a Monday morning is similar to HexB on a Wednesday afternoon. So we only do the split if HexA and HexB are both source_hexes on the same day and time period.

在星期一早上说HexA与在星期三下午说HexB并没有多大意义。 因此，仅当HexA和HexB均为同一日期和时间段的source_hexes时，才进行拆分。

Once the above is done for each pair in the universe, we start building the test control split to ensure that no hex in the test group is also in the control group through some other mapping, as this would contaminate the experiment results.

一旦对Universe中的每个对完成上述操作，我们便开始构建测试控件组，以确保通过其他映射，测试组中的十六进制也不会出现在对照组中，因为这会污染实验结果。

Given that we now have our test-control split, the measure we take is that the test-group source_hexes will have the good_hex included and bad_hex excluded when creating the “dispatch radius”, whereas the control-group will not have the bad_hex excluded and good_hex included. Given that everything else remains the same, the test group should show a reduced ETA compared to the control group post-experiment.

鉴于我们现在已经进行了测试控制拆分，因此我们采取的措施是，在创建“分发半径”时，测试组source_hexes将包含good_hex，而bad_hex将被排除，而控制组将不排除bad_hex，并且包括good_hex。 考虑到其他所有条件均保持不变，与实验后的对照组相比，测试组的ETA应当降低。

We then ran this experiment for 2 weeks and tried to get 1000+ orders cumulatively in both the test group and the control group, so we don’t suffer from data-sparsity while analysing what happened.

然后，我们进行了2周的实验，并试图在测试组和对照组中累计获得1000多个订单，因此在分析发生的情况时，我们不会遭受数据稀疏的困扰。

实验结果 (Experiment Results)

We ran this experiment in Hyderabad where we saw an ETA reduction in test group vs control group of around 9% when comparing the Median ETAs and almost 13% when comparing Mean ETAs. Pre experiment the test and control groups had a difference of only about 3% when looking at both Mean and Median ETAs, thus showing us that the changes we made actually added value to on-ground ETAs.

我们在海得拉巴进行了该实验，与中位数ETA相比，测试组与对照组的ETA降低了约9％ ，而平均ETA则降低了近13％ 。 实验前，测试组和对照组在平均和中位数ETA上的差异仅为3％ ，因此向我们表明，所做的更改实际上为地面ETA增值。

We know that no experiment can be called successful without statistical tests of significance, so we went into the experiment having defined our hypothesis as follows:

我们知道，没有显着性统计检验就不能说成功就是实验，因此我们按照以下假设进行假设的实验：

H0 ( Null Hypothesis ) : Hex based swaps have no effect on realized ETAs

H0(零假设) ：基于十六进制的交换对已实现的ETA无效

H1 ( Alternate Hypothesis ) : Hex based swaps DO have an effect on realized ETAs

H1(备用假设) ：基于十六进制的互换确实会影响已实现的ETA

Rejecting the null hypothesis at a significance level of above 95% is the gold standard that we were striving for, and we are happy to report that we achieved statistically significant results at a level of around 98%, with p-values in the 0.01 range when using a few statistical tests of significance.

我们一直追求的金标准是拒绝高于95％的显着性水平的零假设，并且我们很高兴地报告我们在98％左右的水平上取得了统计学上显着的结果，p值在0.01范围内在使用一些有意义的统计检验时。

When viewed visually, what we got was something similar to this :

从视觉上看，我们得到的类似于以下内容：

What this image is telling us, is that when viewed on a relative scale and after adjusting for pre-experiment ETA delta, we have shifted the center of the test group ETA distribution towards the lower side when compared to the control ETA group, thus showing that our changes have made an impact in lowering ETAs as we expected.

这张图片告诉我们的是，以相对比例查看并调整了实验前的ETA增量后，与对照组ETA组相比，我们已将测试组ETA分布的中心向下方移动，我们的更改对降低ETA产生了预期的影响。

结论 (Conclusion)

The high-level goal as mentioned at the start was to improve a key aspect of dispatch: ETA. We wanted to add a good amount of value by doing something that was not cost-intensive, rather by doing something that leveraged the technology and information we already had. This is the hallmark of any data-science team, to use common sense and best practices to uncover hidden insights using as simple an approach as possible.

一开始提到的高级目标是改进调度的一个关键方面：ETA。 我们想通过做一些不耗费成本的事情，而是通过利用我们已经拥有的技术和信息，来增加很多价值。 这是任何数据科学团队的标志，可以使用常识和最佳实践以尽可能简单的方法来发现隐藏的见解。

If you enjoyed this blog post, check out what we’ve posted so far over here, and keep an eye out on the same space for some really cool upcoming blogs in the near future. If you have any questions about the problems we face as Data Scientists at Rapido, about transitioning to a start-up after a few years in a different field, or about anything else, please reach out to me on LinkedIn or on siddharth.p@rapido.bike, I look forward to answering any questions!

如果您喜欢这篇博客文章，请查看我们到目前为止在这里发布的内容，并在不久的将来留意相同的空间来关注一些即将发布的非常酷的博客。 如果您对我们在Rapido担任数据科学家时遇到的问题，在其他领域工作几年后要过渡到初创公司或其他任何问题有任何疑问，请通过LinkedIn或siddharth.p @与我联系。 Rapido.bike ，我期待回答任何问题！