Michael Johns, Zhenyu Wang, Bruno Dupont, and Luca Fiaschi

迈克尔·约翰斯，王振宇，布鲁诺·杜邦和卢卡·菲亚斯基

“If you can’t measure it, you can’t manage it, or fix it”

“如果无法衡量，就无法管理或修复它”

–Mike Bloomberg

–麦克·彭博(Mike Bloomberg)

Knowing where to allocate marketing dollars and how much to spend is a perennial business problem. The complexity of modern marketing only adds to this challenge. Contemporary measurement methods rely heavily on data from online web-tracking (cookies) that provide only a limited view of advertising touchpoints in the customer journey and could be further jeopardized by new privacy regulations (Juneau, 2020). To get a comprehensive picture of how well marketing budgets are working requires an approach that can account for online (e.g. search, social media, etc.) and offline (TV, radio, etc.) marketing activities with both direct and indirect effects. Media Mix Modeling (MMM) provides one solution to this problem.

知道在哪里分配营销美元以及花费多少是一个长期的业务问题。 现代营销的复杂性只会增加这一挑战。 当代的测量方法严重依赖于来自在线网络跟踪(cookie)的数据，这些数据仅提供了有限的客户旅程中广告接触点的视图，并且可能会受到新的隐私法规的进一步危害(Juneau，2020) 。 要全面了解营销预算的运作情况，需要一种方法，该方法可以说明具有直接和间接影响的在线(例如搜索，社交媒体等)和离线(电视，广播等)营销活动。 媒体混合建模(MMM)提供了解决此问题的一种方法。

This post describes how we built a Media Mix Model of customer acquisition to optimize a yearly budget in the hundreds of millions of dollars. We describe the model, some of the challenges we faced when building it, and discuss how it is used to guide marketing strategy.

这篇文章介绍了我们如何建立客户获取的媒体混合模型，以优化每年数亿美元的预算。 我们将描述该模型，以及在构建模型时我们面临的一些挑战，并讨论如何将其用于指导营销策略。

什么是营销组合模型(MMM)？ (What is a Marketing Mix Model (MMM)?)

Media mix modeling is a statistical modeling technique for quantifying the effectiveness of advertising on business metrics like new customer acquisitions. MMMs have been in use since the 1960’s (e.g., Borden, 1964) and are common in many industries.

媒体组合建模是一种统计建模技术，用于量化广告在诸如新客户获取等业务指标上的有效性。 MMM自1960年代开始使用(例如Borden，1964)，并且在许多行业中很普遍。

Our MMM is designed to estimate the incremental impact of a marketing channel (think Facebook, podcasting, online display ads) on the number of new subscribers. These estimates can be used to better understand and optimize the efficiency of different allocations of our marketing budget (media mix). The MMM is especially helpful in quantifying the impact of offline channels like television, billboards, or radio advertising, which are difficult to assess using digital measurement solutions.

我们的MMM旨在估算营销渠道(例如Facebook，播客，在线展示广告)对新订户数量的增量影响。 这些估算值可用于更好地理解和优化我们营销预算(媒体组合)不同分配的效率。 MMM在量化脱机频道(如电视，广告牌或广播广告)的影响方面特别有用，这些脱机频道难以使用数字测量解决方案进行评估。

我们如何建立MMM (How we built our MMM)

We developed our model based on the approach described in Jin et al. (2017). They propose using Bayesian methods to build a multivariable regression model with transformations on marketing activity variables (e.g., spending) which account for diminishing returns and lagged effects of impressions. Using Bayesian methods gives us the ability to incorporate our prior knowledge about marketing effects into the model and produce results that are easier to use in practice while being consistent with field experiments such as lift tests.

我们基于Jin等人中描述的方法开发了模型。 (2017) 。 他们建议使用贝叶斯方法建立一个多变量回归模型，对营销活动变量(例如支出)进行转换，从而解释收益递减和印象滞后效应。 使用贝叶斯方法使我们能够将我们对营销效果的先验知识整合到模型中，并产生易于在实践中使用的结果，同时与现场测试(例如提升测试)保持一致。

In the core modeling framework, marketing dollars spent each week in specific channels (e.g., Hulu, Facebook, direct mail, banner ads, etc.) are used to predict the total number of new customers acquired that week. Spend per channel is transformed using a saturation function to capture diminishing returns on advertising. Channels that can have decaying effects over time, like TV, are further transformed with a function to capture such lagged effects. Control variables are included to account for external factors that can also influence the number of customers signing up each week, such as seasonal variation and discount offers in circulation.

在核心建模框架中，每周在特定渠道(例如Hulu，Facebook，直接邮件，横幅广告等)上花费的营销费用用于预测该周获得的新客户总数。 使用饱和度函数转换每个渠道的支出，以捕获广告收益递减的情况。 随时间推移可能具有衰减效果的频道(例如电视)会进一步转换为具有捕获此类滞后效果的功能。 包含控制变量以说明可能也会影响每周签约客户数量的外部因素，例如季节性变化和流通中的折扣优惠。

In statistical terms, the model has the following form:

用统计术语来说，该模型具有以下形式：

where x_mt is the spend value for a marketing channel m in week t. Spend is transformed using a saturation and decay function f( ); β_m is the effect of channel m on customer acquisition; z_ct is the value of control variable c in week t; β_c is the effect of control variable c; e_t is a Normal error term with mean 0 and constant variance.

其中x_mt是第t周内营销渠道m的支出值。 使用饱和度和衰减函数f()转换支出； β_m是渠道m对客户获取的影响； z_ct是第t周中控制变量c的值； β_c是控制变量c的作用； e_t是具有均值0和恒定方差的正态误差项。

This means that the number of new customers acquired each week (y_t) is modeled using weekly marketing spend for the different channels while controlling for factors external to marketing. The model produces a coefficient (β_m) for each channel that represents the number of customers that are acquired for each dollar spent, holding all other spending constant. These coefficients are transformed into the total number of new customers that a channel produces. We can then divide the amount spent on the channel by the estimate of customers acquired to get the incremental customer acquisition cost (iCAC), a standard measure of marketing efficiency.

这意味着，每周的新客户数量( y_t )是使用不同渠道的每周营销支出建模的，同时控制了营销外部的因素。 该模型为每个渠道产生一个系数( β_m )，该系数代表每花费1美元获得的客户数量，而使所有其他花费保持不变。 这些系数转化为渠道产生的新客户总数。 然后，我们可以将花费在渠道上的金额除以获得的客户的估算值，以获得增量的客户获得成本 ( iCAC )，这是营销效率的标准衡量标准。

Traditional statistical models would assume that the relationship between marketing and customer acquisition follows a straight line. This assumption is often too simplistic. The yield from marketing dollars will tend to approach a point of saturation as spending increases. That is, marketing dollars will start to show diminishing returns after reaching a certain spend level. To account for this saturation, marketing spend is transformed using a nonlinear function that can capture the diminishing returns on advertising:

传统的统计模型会假设营销和客户获取之间的关系是一条直线。 这种假设通常过于简单。 随着支出的增加，营销美元的收益将趋于接近饱和点。 也就是说，在达到一定的支出水平之后，营销美元将开始显示收益递减。 为了解决这种饱和问题，使用了可以捕获广告收益递减的非线性函数来转换营销支出：

where μ is the half-saturation point and xₘₜ is the spend value. The shape of this curve, which is determined by the parameter μ, is learned for each channel as part of the model fitting process. Figure 1 illustrates the shape of the logistic function for values of μ, ranging from 1 to 7.

其中， μ是半饱和点， xₘₜ是花费值。 在模型拟合过程中，将为每个通道学习由参数μ确定的曲线形状。 图1说明了μ值从1到7的对数函数的形状。

The ability to learn this curve for each input channel while fitting the model is yet another advantage of Bayesian methods.

贝叶斯方法的另一个优点是能够在拟合模型的同时为每个输入通道学习此曲线。

From a marketing perspective, the channel saturation curve allows us to identify the range of spend where we will get the most “bang” for our marketing dollars. The curve can also be plotted against the estimated CAC,(xₘₜ / βₘ f(xₘₜ)), to help understand how quickly efficiency will decrease as spending increases. This pattern can be seen in Figure 2: the CAC generally increases — an indicator of lower efficiency — as spending increases.

从营销的角度来看，渠道饱和度曲线使我们能够确定支出范围，在该范围内，我们可以最大程度地获得营销收入。 该曲线也可以针对估计CAC绘制，(Xₘₜ/βₘF(xₘₜ))，以帮助理解效率会如何Swift降低的支出增加。 在图2中可以看到这种模式：随着支出的增加，CAC通常会增加，这是效率降低的指标。

In addition to saturation effects, the impact of offline marketing is often distributed over time. Marketing channels with these sorts of decaying effects are further transformed using an adstock function. Advertising adstock is the idea that the impact of advertising peaks at a certain point and continues to have a successively weaker effect for some time period after that peak. We use a geometric decay weight that assumes the effect of advertising peaks at the time of exposure and then decreases over time. The weight function controls the rate of decay through the ɑlpha parameter.

除饱和效应外，离线营销的影响通常会随着时间而分布。 使用adstock函数可以进一步转换具有这种衰减效果的营销渠道。 广告广告资源的想法是，广告的影响在某个点达到顶峰，并在该顶峰后的某个时间段内持续产生较弱的影响。 我们使用几何衰减权重，该权重假设在曝光时出现广告峰值，然后随着时间的推移而降低。 权重函数通过ɑlpha参数控制衰减率。

Figure 3 below illustrates the shape of the adstock function with different assumptions about how slowly the impact of media exposure can decay.

下面的图3展示了adstock函数的形状，并假设了媒体曝光的影响衰减的缓慢程度。

A longer adstock means that the effect of advertising persists longer. The length of the adstock is also learned during the model fitting process. Adstock decay is typical for channels like television or radio — the type of advertising that produces multiple impressions over time. The length of the adstock decay gives us additional insight into how a channel functions. A short decay can suggest a short product consideration time; a long decay suggests that it takes some time before prospective customers decide to adopt the product.

较长的广告资源意味着广告效果会持续更长的时间。 还可以在模型拟合过程中了解广告素材的长度。 广告库存衰减是电视或广播等渠道的典型现象，这种广告类型会随着时间的推移产生多次展示。 广告素材衰减的时间长度使我们对渠道如何发挥作用有了更多的了解。 短暂的衰变可能意味着较短的产品考虑时间。 长时间的衰退表明潜在客户决定采用该产品需要一些时间。

营销渠道会计 (Accounting for the Marketing Funnel)

The mixture of marketing channels is often organized into a funnel. Figure 4 illustrates how we might organize offline and online marketing channels based on our knowledge of the way different types of advertising can influence the customer journey.

营销渠道的混合通常组织成一个渠道。 图4说明了我们如何根据对不同类型广告可能影响客户旅程的方式的了解来组织离线和在线营销渠道。

Offline, brand channels are typically placed at the top of the funnel; online, direct response channels tend to fall at the bottom. The idea is that channels at the top of the funnel create awareness and interest that drives prospective customers to online channels at the bottom of the funnel before converting.

离线品牌频道通常位于渠道顶部； 在线直接响应渠道往往落在底部。 想法是，渠道顶部的渠道可以提高知名度和兴趣度，从而在转化之前将潜在客户吸引到渠道底部的在线渠道。

One implication of the funnel structure is that offline marketing can have a significant downstream effect on certain online channels, a phenomenon referred to as a “funnel effect”. For example, after hearing a podcast ad, a prospective customer would need to go online, where they might encounter search engine marketing. Including channels in the model from all levels of the funnel implies that the amount spent in each channel can be equally impactful in driving conversions. In reality, the amount spent on channels like Search Engine Marketing is more often a product of spending on brand marketing higher up in the funnel (e.g. on TV). Failing to consider these hierarchical relationships in the model could produce misleading results.

渠道结构的一个含义是，离线营销可以对某些在线渠道产生重大的下游影响，这种现象被称为“渠道效应”。 例如，在听到播客广告后，潜在客户可能需要上线，在那里他们可能会遇到搜索引擎营销。 在渠道的各个级别中将模型中的渠道包括在内，意味着在每个渠道中花费的金额对推动转化的影响同等。 实际上，在搜索引擎营销等渠道上花费的金额通常是渠道(例如电视)上的品牌营销支出的产物。 不考虑模型中的这些层次关系可能会产生误导性的结果。

To deal with these potential funnel effects we estimate the efficiency of marketing spend in two steps. In the first step we fit the core MMM (described above and depicted in Figure 5) while excluding spend from a set of mediating channels at the bottom of the funnel: search engine marketing, affiliate marketing, and online leads. Leaving these downstream channels out of the model gives channels further up in the funnel the opportunity to get credit for the downstream impacts that would otherwise be obscured. We call this the direct model.

为了处理这些潜在的渠道效应，我们分两步估算营销支出的效率。 第一步，我们调整核心MMM(如上所述并在图5中进行了描述)，同时从渠道底部的一组中介渠道( 搜索引擎营销 ， 会员营销和在线 销售 线索 )中排除了支出。 将这些下游渠道排除在模型之外，可以使渠道中更远的渠道有机会获得本来可以避免的下游影响的信誉。 我们称其为直接模型 。

In the second step, we then use the same set of marketing channels to model the amount of weekly spend (not conversions) for search engine marketing, affiliate marketing, and online leads. The coefficients from this lower-funnel model tell us how much marketing spend in the “top” of the funnel is absorbed by these channels at the “bottom” of the funnel. We call this the undirect model and is depicted in Figure 6.

在第二步中，我们然后使用同一组营销渠道来模拟每周支出金额(而非转化) 用于搜索引擎营销，会员营销和在线潜在客户 。 这个较低渠道模型的系数告诉我们，渠道“顶部”的渠道吸收了渠道“顶部”的营销支出。 我们称其为非直接模型 ， 如图6所示。

Spend estimates from the directed model are then corrected using the undirected one, producing a funnel-adjusted CAC. Calculating the CAC using the adjusted spend provides a more accurate picture of each channel’s marketing efficiency. Figure 6 shows how the CACs can shift following adjustment for lower-funnel impacts.

然后，使用无向模型校正有向模型的支出估算值，从而产生漏斗调整后的CAC。 使用调整后的支出计算CAC，可以更准确地了解每个渠道的营销效率。 图6显示了CAC如何针对调整后的漏斗影响而转移。

The mathematical formulation of our fully adjusted CAC* for each time-step t and upper-funnel channel m is:

对于每个时间步长t和上漏斗通道m ，我们完全调整的CAC *的数学公式为：

Where η_mt is a spend adjustment obtained by calculating the effect of the upper funnel channel on the lower funnel spend. And ξ_mt is a seasonal adjustment to conversions obtained by redistributing the extra conversions from seasonal variables. These coefficients are derived from the models for total conversions (y_t, direct model) and total lower funnel spend (undirect model) as follows:

其中η_mt 是通过计算上层渠道渠道对下层渠道支出的影响而获得的支出调整。 和ξ_mt 是对转化的季节性调整，该转化是通过重新分配季节性变量的额外转化获得的。 这些系数是从总转化模型( y_t ，直接模型)和总漏斗支出总额( 非直接模型) 得出的 ，如下所示：

使模型可解释且可行 (Making the Model Interpretable and Actionable)

A major goal that guided our approach was ensuring that the results would be plausible, interpretable, and actionable. Outputs from inferential models — models that describe relationships — can be difficult to understand and apply in practice. Using a Bayesian modeling approach provides several advantages in accomplishing this goal.

指导我们方法的一个主要目标是确保结果是合理的，可解释的和可操作的 。 推论模型(描述关系的模型)的输出可能难以理解并难以在实践中应用。 使用贝叶斯建模方法可在实现此目标方面提供多个优势。

Fitting a Bayesian model requires first specifying a prior on the model coefficient being estimated. The prior defines a range of possible values for the channel estimate and a proposal for how likely those values are to occur. It is essentially an educated guess about how much a particular channel influences conversions. The model then uses the observed marketing data to update the proposal encoded in the prior. The most likely value after updating the priors is our best estimate for the marketing channel coefficient.

拟合贝叶斯模型需要首先指定要估计的模型系数的先验 。 先验定义了用于信道估计的可能值的范围以及关于这些值出现的可能性的建议。 从本质上来说，这是对某个特定渠道会影响转化的有根据的猜测。 然后，该模型使用观察到的营销数据来更新先前编码的提案。 更新先验后最有可能的价值是我们对营销渠道系数的最佳估计。

Specifying a prior for each channel estimate makes it possible to set constraints on the results to ensure they fall within a sensible range. For example, it is highly unlikely that marketing would ever have a negative impact on customer acquisition. We, therefore, constrain all channel estimates to be non-negative.

为每个通道估计指定一个先验，可以对结果设置约束以确保它们落在合理范围内。 例如，营销不太可能对客户获取产生负面影响。 因此，我们将所有通道估计约束为非负值。

With the prior, we can also use information from lift tests or incrementality analyses to tune model estimates and make them consistent with external benchmarks. Once built, the model is then continually evaluated and adjusted as we conduct tests and get new information about the effectiveness of our marketing. This makes the MMM extremely adaptable to changes in the marketing landscape. We can also incorporate feedback from channel managers to adjust estimates that are unrealistically large or small. As a result, we can provide outputs that are consistent with existing knowledge and responsive to stakeholder needs.

借助先前版本，我们还可以使用提升测试或增量分析中的信息来调整模型估算值，并使它们与外部基准保持一致。 建立模型后，我们将在进行测试并获得有关营销有效性的新信息时，对模型进行持续评估和调整。 这使得MMM非常适应营销环境的变化。 我们还可以结合渠道经理的反馈来调整不切实际的大小估算值。 结果，我们可以提供与现有知识一致并响应利益相关者需求的输出。

在PyMC3中构建贝叶斯MMM (Building a Bayesian MMM in PyMC3)

The sample code below illustrates how to implement a simple MMM with priors and transformation functions using PyMC3. For this toy example, we assume that there are three marketing channels (X1, X2, X3) and one control variable (Z1). Each marketing channel is transformed using a saturation function to model diminishing returns.

以下示例代码说明了如何使用PyMC3通过先验和转换功能实现简单的MMM。 对于此玩具示例，我们假设存在三个营销渠道(X1，X2，X3)和一个控制变量( Z1 )。 使用饱和度函数对每个营销渠道进行转换，以建立收益递减模型。

Building a model requires first defining priors on the unknown parameters to be estimated: intercept; marketing channel, beta; control variable, c_beta; saturation curve shape parameter, half_sat; likelihood noise, sigma. Figure 7 shows the shape of key prior distributions, which are based on the recommendations of Jin et al. (2017).

构建模型要求首先的未知参数先验定义要估计： 截距 ; 营销渠道， 测试版 ； 控制变量c_beta ; 饱和曲线形状参数half_sat ; 似然噪声， sigma 。 图7显示了关键先验分布的形状，这些分布基于Jin等人的建议。 (2017)。

We next define the model of the expected number of activations, mu, as a linear combination of baseline conversions, the spend in the marketing channels, and the impact of the control variable. This model defines the mean of the likelihood function for the outcome, y_obs, which is represented as a normal distribution with mean μ and variance σ in a linear regression model. Once the model is defined, we sample from the resulting probability space to produce the posterior distributions of the model parameters. Figure 8 shows a sample trace plot that displays the history of the sampling process for a model parameter.

接下来，我们将预期激活次数mu的模型定义为基线转化，营销渠道支出和控制变量影响的线性组合。 该模型定义了结果的似然函数y_obs的平均值，在线性回归模型中以平均值μ和方差σ表示为正态分布。 定义模型后，我们将从结果概率空间中采样以产生模型参数的后验分布。 图8显示了一个示例轨迹图，该轨迹图显示了模型参数的采样过程的历史记录。

The inspection of the trace plot is used to confirm that the sampling process was able to effectively explore the probability space and converge on a valid distribution. Additionally, it is typical to check model performance by conducting a posterior predictive check (PPC). Data are simulated from the fitted model and compared to the observed data. Variation in the simulated data can be used to build plausibility intervals that are similar in spirit to a confidence interval. Figure 9 shows a sample PPC with a 95% plausibility interval for predicted and observed data.

跟踪图的检查用于确认采样过程能够有效地探索概率空间并收敛于有效分布。 此外，通常通过进行后验预测检查 (PPC)来检查模型性能。 从拟合模型模拟数据，并将其与观察到的数据进行比较。 模拟数据的变化可用于建立在本质上与置信区间相似的合理区间。 图9显示了具有95％可信区间的样本PPC，用于预测和观察的数据。

Inferences for the target variable and the parameters can then be extracted using the mean (or median) and area of highest probability density of the posterior distributions.

然后可以使用后验分布的均值(或中位数)和最高概率密度区域来提取目标变量和参数的推论。

使用MMM (Putting the MMM to Use)

MMMs have several applications. As noted, a common use is quantifying the effectiveness and efficiency of marketing. This is particularly important for monitoring offline channels that are difficult to measure on a continuous basis. It also helps us identify lower funnel online channels that tend to absorb some of the influence of upper funnel channels. Taken as a whole, the model gives us an integrated view of channel performance and inter-channel dynamics.

MMM具有多个应用程序。 如前所述，通常的用途是量化营销的有效性和效率。 这对于监视难以连续测量的脱机频道特别重要。 它还有助于我们确定下层渠道在线渠道，这些渠道倾向于吸收上层渠道的某些影响。 总体而言，该模型为我们提供了渠道性能和渠道间动态的综合视图。

While an MMM can help fill knowledge gaps, the overarching purpose of the model is to provide insights that can inform strategic planning. We use iCAC estimates from the model to identify channels that are underperforming, as well as those that are highly efficient. Spend can then be shifted between channels to produce an overall mix that maximizes the total efficiency of our marketing.

尽管MMM可以帮助填补知识空白，但是该模型的总体目的是提供可以为战略规划提供依据的见解。 我们使用模型中的iCAC估算值来确定效果不佳的渠道以及高效渠道。 然后可以在渠道之间转移支出，以产生整体组合，从而最大限度地提高营销的整体效率。

To do this, we plug model coefficients into a constrained optimization algorithm to generate recommendations for how to distribute our marketing budget, B, to maximize the number of conversions in a certain period of time:

为此，我们将模型系数插入受约束的优化算法中，以生成有关如何分配营销预算B的建议 ，以在特定时间段内最大化转化次数：

The optimization routine uses a nonlinear gradient method to find the spend levels for our channels that will produce the lowest blended CAC (i.e., total spend / total conversion), given constraints on overall and channel-level budgets (defined by the two b_mt constants and the value of B). Constraints are discussed with marketing stakeholders to ensure optimization is performed under realistic conditions and results are actionable. By comparing the algorithmic recommendations to historical spending patterns we can identify new channel development opportunities that might go unnoticed when looking at a single channel in isolation.

优化程序使用非线性梯度方法来查找我们的渠道的支出水平，该水平将产生最低的混合 CAC(即总支出/总转化 )，这要考虑到总体预算和渠道预算(由两个b_mt常数和B的值)。 与市场利益相关者讨论约束条件，以确保在现实条件下执行优化，并且结果是可行的。 通过将算法建议与历史支出模式进行比较，我们可以确定在单独查看单个渠道时可能不会注意到的新渠道开发机会。

结论 (Conclusion)

Media mix modeling is a powerful tool for measuring and managing a complex marketing mix. By accounting for marketing spend saturation, advertising decay, and the marketing funnel hierarchy, the MMM offers a flexible tool for evaluating the performance of both online and offline marketing channels. Moreover, using a Bayesian framework provides us with the ability to incorporate existing marketing knowledge into the model. The MMM will continue to evolve in form and function as it is calibrated based on lift tests of various marketing channels. In the meantime, we will also keep exploring other areas where this model can be helpful, such as demand forecasting and business planning.

媒体组合建模是衡量和管理复杂营销组合的强大工具。 通过考虑营销支出饱和度，广告衰减和营销渠道层次结构，MMM提供了一种灵活的工具来评估在线和离线营销渠道的效果。 此外，使用贝叶斯框架使我们能够将现有的营销知识整合到模型中。 MMM将根据各种营销渠道的提升测试进行校准，因此其形式和功能将继续发展。 同时，我们还将继续探索该模型可能有帮助的其他领域，例如需求预测和业务计划。

[1] Borden, N. H. (1964). The concept of the marketing mix. Journal of advertising research, 4, 2–7.

[1] Borden，NH(1964)。 营销组合的概念 。 广告研究杂志，4，2–7 。

[2] Jin, Y., Wang, W., Sun, Y., Chan, D., & Koehler, J. (2017). Bayesian methods for media mix modeling with carryover and shape effects. Google Research.

[2] Jin，Y.，Wang，W.，Sun，Y.，Chan，D.，＆Koehler，J.(2017)。 具有残留和形状效应的贝叶斯媒体混合建模方法 。 Google研究。

[3] Juneau, T. (2020). Digital Marketing In a Cookie Less Internet. Forbes.

[3]朱诺，T。(2020)。 少Cookie的Internet中的数字营销 。 福布斯

[4] Chan, D., Perry, M. (2017) Challenges And Opportunities In Media Mix Modeling, Google Research.

[4] Chan D.，Perry，M.(2017) 媒体混合建模的挑战与机遇 ，谷歌研究。