disclaimer: I don’t work for Yammer, this is a public data case study, I’ve written it in a narrative format to make this case study more engaging to read.

免责声明：我不为Yammer工作，这是一个公共数据案例研究，我以叙述性格式编写了该案例研究，以使该案例研究更具阅读吸引力。

It’s Monday morning…you sit down at your desk with a cup of coffee — your eyes barely open. Suddenly the head of product taps you on your shoulder and slams his laptop down on your desk.

今天是星期一早上……您坐在办公桌旁喝咖啡，几乎睁不开眼睛。 突然产品的头部轻拍您的肩膀，将笔记本电脑砸向您的书桌。

“How did our weekly engagement drop 21% in a month?!”

“我们的每周参与度如何在一个月内下降21％？！”

*Sighhhhhhh*

* Sighhhhhhh *

You crack your knuckles, and put down your coffee — “let’s take a look”.

您打开指关节，放下咖啡-“让我们看一下”。

Hmmm…at first glance it looks our weekly engagement levels have been steadily declining since July 28. At Yammer, we define ‘engagement’ as any interaction with the product that makes a “server call” (aka if the user clicked on something).

嗯……乍看之下，自7月28日以来，我们的每周参与度一直在稳步下降。在Yammer，我们将“参与度”定义为与产品进行“服务器呼叫”的任何互动(也就是用户点击了某项内容)。

In general, most companies look at user engagement as one of their main KPIs (key performance indicators) for overall growth & health of their product.

通常，大多数公司将用户参与度视为产品整体增长和健康状况的主要KPI(关键绩效指标)之一。

To those who haven’t heard of us, Yammer is a social network for enterprises (similar to Workplace by Facebook). Employees can send messages, search the intranet, share documents, and post updates.

对于从未听说过我们的人来说， Yammer是企业的社交网络(类似于Facebook的Workplace)。 员工可以发送消息，搜索Intranet，共享文档以及发布更新。

诊断问题 (Diagnosing the Problem)

Before spending too much time diagnosing the root cause for a KPI shift, I first go through this mental checklist to ensure the root cause wasn’t caused by non-product factors such as advertising, bugs, errors, etc.

在花太多时间诊断KPI变动的根本原因之前，我首先查看此心理检查清单，以确保根本原因不是由非产品因素引起的，例如广告，错误，错误等。

Time: Time is a good factor to consider since irregular trends are easier to spot when graphed. Here are the most common time-related data questions to ask yourself:

时间 ：时间是一个值得考虑的好因素，因为在绘制图形时不规则趋势更容易发现。 以下是最常见的与时间相关的数据问题：

• Is this drop in engagement sudden or gradual? ⟶ If it’s sudden then check with the engineering team if any deployments were made, if its gradual then it may be related to a change in user behaviour.
  参与度下降是突然的还是逐渐的？ ⟶如果突然，请咨询工程团队是否进行了任何部署，如果部署是逐步的，则可能与用户行为的更改有关。
• Is it a one time occurrence or a recurring issue? ⟶ a one time occurrence is likely due to a technical issue or marketing campaign while recurring issues could be due to behavioural changes.
  是一次性发生还是反复出现的问题？ one一次发生可能是由于技术问题或市场营销活动，而经常发生的问题可能是由于行为更改。
• Do we see specific times during the day or specific days of the week where the drop is more pronounced? ⟶ Drops localized to a specific time or day would likely indicate a technical issue — such as failed data refreshes that generally occur over the weekend.
  在一天中或一周中的特定几天中，我们看到特定的时间了吗？ local本地化到特定时间或日期的丢弃可能表明存在技术问题，例如通常在周末发生的数据刷新失败。

Region: Regional engagement levels are important to consider since companies may roll out features with specific regions when testing new products or regional internet censorship/ laws that can limit usage. Some questions to ask are:

地区 ：考虑区域参与度很重要，因为公司在测试新产品或可能限制使用的区域互联网审查/法律时可能会在特定区域推出功能。 要问的一些问题是：

• Do we see the drop in engagement tied to a particular region or is it globally distributed? Certain features might perform better or worse in specific locales due to cultural differences in consumer behaviour.
  我们是否看到与特定区域相关的参与度下降？还是全球分布？ 由于消费者行为的文化差异，某些功能在特定区域可能会表现得更好或更差。
• Have there been any recent changes in internet censorship laws related to the region with drops in engagement?
  由于参与度下降，与该地区相关的互联网审查法律最近是否发生了变化？

Other Features / Products: keeping general tabs on other products/features within the same space is useful for identifying potential changes at a company level. Some questions to ask are:

其他功能/产品 ：将其他产品/功能的常规选项卡放在同一空间内对于识别公司级别的潜在更改很有用。 要问的一些问题是：

• Are other features or products also experiencing similar drops in engagement? ⟶ if the drop is prevalent across other products and features, then the issue would be a larger problem that requires involvement from multiple teams to investigate.
  其他功能或产品的参与度是否也在下降？ ⟶如果下降趋势在其他产品和功能中普遍存在，那么问题将是一个更大的问题，需要多个团队的参与才能进行调查。
• Are other similar features or products within the ecosystem experiencing a proportional increase in engagement? ⟶ users could switch to using a different product or feature to fulfil their use-cases leading to cannibalization of Yammer.
  生态系统中的其他类似功能或产品是否正在按比例增加参与度？ ⟶用户可以切换到使用其他产品或功能来满足他们的用例，从而使Yammer蚕食。

Platform: depending on the platform users engage with Yammer, there are differences in UX display and ENG release processes, you want to narrow down if the issue is technical and isolated to a particular platform. Some common questions to ask are:

平台 ：根据使用Yammer的平台用户的不同，UX显示和ENG发布过程有所不同，如果问题是技术性的且仅针对特定平台，则希望缩小范围。 一些常见的问题是：

• Do we see the decline in occurring across specific platforms (ie. mobile, desktop, tablet) or across all platforms?
  我们是否看到在特定平台(即移动，台式机，平板电脑)或所有平台上出现的下降？
• Is the decline android or iOS or other operating system specific?
  衰退的android或iOS或其他操作系统是否特定？

Industry/ Competitors: keeping track of how the Yammer is performing in the industry and having knowledge of general trends is useful for market space awareness. Competitor information may be difficult to acquire, but we can still leverage news articles, google trend searches and third party data to derive high-level insights.

行业/竞争对手 ：跟踪Yammer在行业中的表现并了解总体趋势有助于提高市场空间意识。 竞争对手的信息可能很难获取，但是我们仍然可以利用新闻报道，谷歌趋势搜索和第三方数据来获得高级见解。

Broken Feature / Tracking Code: It is also possible that certain features or the tracking code is broken which affects how events are logged. The mapping hierarchy of events that a user triggers when they interact with the application is important to determine where a user is dropping off.

功能/跟踪代码损坏 ：某些功能或跟踪代码也可能损坏，这会影响事件的记录方式。 用户在与应用程序进行交互时触发的事件的映射层次结构对于确定用户在何处下车很重要。

Bots and Search Crawler Engines: most major websites have high activity from bots but it is difficult to determine the events triggered by bots. High traffic websites are also guided by SEO and changes to search engine indexes can cause significant changes in traffic.

机器人程序和搜索爬虫引擎 ：大多数主要网站的机器人活动都很活跃，但很难确定由机器人程序触发的事件。 高流量的网站也受到SEO的引导，搜索引擎索引的更改可能会导致流量的重大变化。

Major Events: it is also possible for one-off events to cause influxes or major decreases in engagement such as holidays, successful campaigns and negative press.

重大事件 ：一次性事件也可能导致大量涌入或参与度大幅下降，例如假期，成功的竞选活动和负面新闻。

数据约束 (Constraints of the Data)

In this case study, the only datasets we have available to manipulate are: time, region, platforms and event tracking. If you’d like to see the source data and code for analysis please see the embedded notebook link below:

在此案例研究中，我们唯一可操作的数据集是：时间，区域，平台和事件跟踪。 如果您想查看源数据和代码以进行分析，请参见下面的嵌入式笔记本链接：

*Note that all Plotly charts are interactive, you can click and drag to zoom in the chart or click categories in the legend to filter.

*请注意，所有Plotly图表都是交互式的，您可以单击并拖动以放大图表，也可以单击图例中的类别进行过滤。

假设 (Hypotheses)

Potential Cause 1: engagement dropped due to failed data refreshes or technical outages on a specific week day or time of day

可能的原因1 ：由于在特定的工作日或一天中的某天数据刷新失败或技术中断而 导致 参与度下降

• As per the graphs above, Saturdays and Sundays have the lowest engagement rates which makes sense since Yammer is used mostly for work.

  根据上面的图表，周六和周日的参与率最低，这是有道理的，因为Yammer主要用于工作。

• When checking engagement by Time of Day (based on the 24-hour clock) it looks like 9 am is when users interact with Yammer the most frequently but no other time of day stands out.
  当按一天中的时间(基于24小时制)检查参与度时，用户与Yammer进行互动的频率最高，但一天中其他时间都没有，这似乎是上午9点。
• Since the drop in engagement was gradual over the span of a few months and not localized to a particular day or time we can assume that the cause is not likely due to a one time server / technical outage or data refresh failure.
  由于参与度的下降是在几个月的时间内逐渐发生的，并且没有局限于特定的日期或时间，因此我们可以假定原因可能不是服务器/技术故障或数据刷新失败一次。

Potential Cause 2: decrease in engagement due to regional outages or locale based feature testing

潜在原因2 ：由于区域中断或基于区域设置的功能测试而导致参与度降低

• Since Yammer is used by multiple regions, it would be best to analyze the top 5 regions based on number of users.
  由于Yammer被多个区域使用，因此最好根据用户数量来分析前5个区域。
• From the graph above, the United States had the most significant drop in active users while other regions have experienced smaller decreases.
  从上图可以看出，美国的活跃用户下降幅度最大，而其他地区的下降幅度较小。
• It would be useful to quickly check with other teams if there were any new releases or product upgrades localized to only the States.
  快速与其他团队核对是否有仅本地化的新版本或产品升级会很有用。

Potential Cause 3: decrease in engagement is because of failing to acquire and grow users over time — potentially due to tracking errors or bugs interrupting the sign-up flow or the product not providing the right value to users

潜在原因3 ：参与度下降是由于无法随着时间的推移获取和增长用户而引起的-可能是由于跟踪错误或错误干扰了注册流程，或者产品未能为用户提供正确的价值

• Events (actions) in Yammer are put into the following categories: login events, messaging events, search events and sign up funnel events.
  Yammer中的事件(操作)分为以下几类：登录事件，消息事件，搜索事件和注册漏斗事件。
• From the graph above, it is clear that users are able to navigate through the signup funnel, but other events are seeing a decrease in engagement levels.
  从上图可以明显看出，用户可以浏览注册渠道，但其他事件的参与度却有所下降。

• Growth or activation rate is a closely tracked metric for all companies — since it means Yammer is providing the intended value for new customers.
  增长率或激活率是所有公司密切跟踪的指标-因为这意味着Yammer为新客户提供了预期的价值。
• Based on the chart above, the growth rate remains normal as it continues to be high during the weekday and low on weekends.
  根据上面的图表，增长率保持正常，因为它在工作日中继续保持较高水平，而周末则保持较低水平。
• With no sign up (growth) issues, the drop in engagement can be assumed to be from older / existing users.
  在没有注册(增长)问题的情况下，参与度下降的原因可以认为是老用户/现有用户。

Potential Cause 4: decrease in engagement due to lack of retention from older users

潜在原因4 ：参与度下降是由于缺少老年用户的保留

• Doing a cohort analysis is one of the most common ways to track retention for any group of users. It helps to paint a better picture about Yammer’s product effectiveness for users in the long run.

  进行同类群组分析是跟踪任何一组用户的保留率的最常见方法之一。 从长远来看，它有助于为用户更好地描绘Yammer产品的有效性。

• After segmenting the users by age, it’s quite evident that those who signed up more than 10 weeks prior to May 1 have decreasing levels of engagement over time. Looking at each cohort individually, as the weeks progress, their engagement levels also decrease.
  在按年龄对用户进行细分之后，很明显，那些在5月1日之前签约超过10周的用户的参与度会随着时间的推移而下降。 逐个查看每个群组，随着星期的进展，他们的参与度也会降低。
• As the problem is related to mature users, the issue is unlikely to be related to a one-time spike from marketing traffic or bots and search crawlers — which can cause unsustainable engagement blitzs.
  由于该问题与成熟用户有关，因此该问题不太可能与来自营销流量或漫游器和搜索爬虫的一次性高峰有关，这可能会导致不可持续的用户参与度。

• The product has a problem with “stickiness”, since it seems like our users become less engaged with our product over time.

  该产品存在“ 粘性 ”问题，因为随着时间的流逝，我们的用户似乎越来越不喜欢我们的产品。

Potential Cause 5: decrease in engagement due to our weekly digest emails failing to reach users for its intended purpose.

潜在原因5 ：由于我们的每周摘要电子邮件未能达到其预期目的而无法吸引用户，因此参与度下降。

• To engage and re-engage users, Yammer sends weekly digest emails and weekly re-engagement emails.
  为了吸引和重新吸引用户，Yammer每周发送摘要电子邮件和每周重新参与电子邮件。
• Based on the charts above, the number of weekly emails sent and weekly emails opened have been increasing but there was a significant drop in the number of weekly click throughs while the weekly re-engagement metrics were normal.
  根据上面的图表，每周发送的电子邮件和打开的每周电子邮件的数量一直在增加，但是每周的点击率却有明显下降，而每周的重新参与度指标是正常的。

• A decrease in click through rates even though open rates are increasing could mean that the weekly digest email content is not relevant enough to users or the intended user action is not explicit enough. IF CTR was 0, that would have indicated a broken button or feature.
  即使打开率增加，点击率也会降低，这可能意味着每周摘要电子邮件内容与用户的相关性不够，或者预期的用户操作不够明确。 如果CTR为0，则表明按钮或功能已损坏。

Potential Cause 6: decrease in engagement due to the type of device users use to navigate Yammer or the operating system.

潜在原因6 ：由于用户使用导航Yammer或操作系统的设备类型减少了参与度。

• As seen in the graphs above, users who use phones are seeing a much more pronounced drop in engagement
  如上图所示，使用电话的用户的参与度下降更为明显
• This could be due to the recent changes or releases for Yammer’s mobile app
  这可能是由于Yammer的移动应用程序最近发生了更改或发布

• Digging further for sanity, it is useful to compare the same metric between different operating systems since there are different code versions for the app per OS
  进一步挖掘合理性，比较不同操作系统之间的同一指标非常有用，因为每个操作系统的应用程序代码版本不同
• It looks like regardless of the operating system, users who access Yammer via a phone have been less engaged over time
  不管使用什么操作系统，通过电话访问Yammer的用户随着时间的推移似乎很少参与
• This scenario leads me to assume that there is an issue with retaining Yammer phone users over time, potentially due to UX design problems or dysfunctional features leading to discontent.
  这种情况使我假设，随着时间的流逝，保留Yammer电话用户存在问题，这可能是由于UX设计问题或功能失调导致不满。

结论 (Conclusion)

From our analysis, we can infer that long term mobile users who access Yammer become disengaged over time due to usability issues or irrelevant content. Despite relatively high email open rates, we have observed much lower click through rates from the weekly digests. This likely means there’s an issue with our email content e.g. relevance or quality.

根据我们的分析，我们可以推断出，由于可用性问题或不相关的内容，访问Yammer的长期移动用户会随着时间的流逝而失去联系。 尽管电子邮件的打开率相对较高，但我们发现每周摘要的点击率要低得多。 这可能意味着我们的电子邮件内容存在问题，例如相关性或质量。

Given the multiple different hypotheses for user engagement drop-off, I would discuss validation strategies with other disciplines such as engineering, design, marketing, and product. Validation strategies include, AB testing, feature roll-backs, and locale segmentation.

考虑到用户参与度下降的多种不同假设，我将与工程，设计，营销和产品等其他学科讨论验证策略。 验证策略包括AB测试，功能回滚和区域划分。

使用过的技术的学习 (Learnings with Technologies Used)

• Since I began my data journey with SQL, I’m much more comfortable doing joins and aggregations with SQL than Pandas. I decided to use SQLite for all of the more complicated data manipulations and found that most of the functions and syntax is the same as MySQL. The major differences in my use case is with the date and time functions. I would recommend for individuals to use SQLite to query data frames with SQL syntax if that is your preferred language of choice.

  自从我开始使用SQL进行数据之旅以来，与Pandas相比，使用SQL进行连接和聚合要舒适得多。 我决定将SQLite用于所有更复杂的数据操作，并发现大多数功能和语法与MySQL相同。 我的用例的主要区别在于日期和时间函数 。 我建议个人使用SQLite来查询SQL语法的数据帧(如果这是您的首选语言)。

• I used Plotly’s Python graphing library for all of the visualizations and interactive charts/ graphs. I like the interactive components: hover, click and drag to zoom into elements and the simple layout of Plotly charts. It is my preferred visualization library it as opposed to Matplotlib, Seaborn and Bokeh. I find it a little difficult to find case specific examples online for how to plot certain graphs so I am uncertain about whether I’ve plotted the line and bar charts in the most efficient manner — love to hear some feedback on the code written.
  我将Plotly的Python图形库用于所有可视化效果和交互式图表/图形。 我喜欢交互式组件：悬停，单击并拖动以放大元素和Plotly图表的简单布局。 与Matplotlib，Seaborn和Bokeh相比，它是我首选的可视化库。 我发现很难在网上找到特定案例的示例来绘制某些图形，因此我不确定我是否以最有效的方式绘制了折线图和条形图-喜欢听到对编写代码的一些反馈。

• Plotly’s interactive charts also don’t render in GitHub. If you want to retain your charts to share the analysis in your Jupyter Notebook / Labs, i’d recommend using nbviewer to render your published notebook.

  Plotly的交互式图表也不在GitHub中呈现。 如果要保留图表以在Jupyter笔记本/实验室中共享分析，我建议使用nbviewer渲染已发布的笔记本。

You can find the original case here: https://mode.com/sql-tutorial/a-drop-in-user-engagement/

您可以在这里找到原始案例： https : //mode.com/sql-tutorial/a-drop-in-user-engagement/