CDH使用秘籍(一):Cloudera Manager和Managed Service的数据库

背景

从业务发展需求,大数据平台须要使用spark作为机器学习、数据挖掘、实时计算等工作,所以决定使用Cloudera Manager5.2.0版本号和CDH5。
曾经搭建过Cloudera Manager4.8.2和CDH4,在搭建Cloudera Manager5.2.0版本号的时候,发现对应的Service Host Monitor 和 Service Monitor不能配置外部表,刚开是还以为是配置出错,后来才发现应该是新版本号的Cloudera的存储改变方式了。查了非常多文档,果然发现,新版本号中Service Host Monitor 和 ServicMonitore 不须要配置数据库,默认使用内置存储方式。而且不能改动

概述

Cloudera Manager uses databases to store information about the Cloudera Manager configuration, as well as information such as the health of the system or task progress. For quick, simple installations, Cloudera Manager can install and configure an embedded PostgreSQL database as part of the Cloudera Manager installation process. In addition, some CDH services use databases and are automatically configured to use a default database. If you plan to use the embedded and default databases provided during the Cloudera Manager installation, see Installation Path A - Automated Installation by Cloudera Manager.

Although the embedded database is useful for getting started quickly, you can also use your own PostgreSQL, MySQL, or Oracle database for the Cloudera Manager Server and services that use databases.

须要的数据库
The Cloudera Manager Server, Activity Monitor, Reports Manager, Hive Metastore, Sentry Server, Cloudera Navigator Audit Server, and Cloudera Navigator Metadata Server all require databases. The type of data contained in the databases and their estimated sizes are as follows:
  • Cloudera Manager - Contains all the information about services you have configured and their role assignments, all configuration history, commands, users, and running processes. This relatively small database (<100 MB) is the most important to back up.
  • Activity Monitor - Contains information about past activities. In large clusters, this database can grow large. Configuring an Activity Monitor database is only necessary if a MapReduce service is deployed.
  • Reports Manager - Tracks disk utilization and processing activities over time. Medium-sized.
  • Hive Metastore - Contains Hive metadata. Relatively small.
  • Sentry Server - Contains authorization metadata. Relatively small.
  • Cloudera Navigator Audit Server - Contains auditing information. In large clusters, this database can grow large.
  • Cloudera Navigator Metadata Server - Contains authorization, policies, and audit report metadata. Relatively small.

The Cloudera Manager Service Host Monitor and Service Monitor roles have an internal datastore(注意。就是此处说明了, Host Monitor and Service Monitor在CM5版本号中,不能配置外部表,仅仅能使用内置表。

与CM4版本号有差别)


Cloudera Manager 提供三种不同的安装方式,方法A是自己主动化安装。方法B和C是使用rpm或tar手动安装:
  • Path A automatically installs an embedded PostgreSQL database to meet the requirements of the services. This path reduces the number of installation tasks to complete and choices to make. In Path A you can optionally choose to create external databases forActivity Monitor, Reports Manager, Hive Metastore, Sentry Server, Cloudera Navigator Audit Server, and Cloudera Navigator Metadata Server.
  • Path B and Path C require you to create databases for the Cloudera Manager Server, Activity Monitor, Reports Manager, Hive Metastore, Sentry Server, Cloudera Navigator Audit Server, and Cloudera Navigator Metadata Server.

使用外部数据库须要很多其它的输入以及相关工作,可是cloudera提供了很多其它的兼容性和扩展性,让你能够弹性的选择数据库和配置。

当然能够在一套系统中安装多种不同的数据库。可是这样会带来非常多不确定的因素。所以cloudera建议始终使用同一种数据库。

在非常多样例中,你须要将对应的service与database安装到同一台机器上,能够减小网络IO。提高总体效率。
当然,你也能够将service和database分开安装到不同的机器上。在大型部署中或者database管理员须要这种配置,比方这种场景,Oracle DBA须要独立的管理database。

搭建数据库的配置參考官网。有具体配置步骤:
搭建Cloudera Manager Server数据库
为Activity Monitor, Reports Manager, Hive Metastore, Sentry Server, Cloudera Navigator Audit Server, and Cloudera Navigator Metadata Server搭建外部数据库
为Hue。Oozie搭建外部数据库

下一篇文章中,我将具体介绍Cloudera Manager中database的存储机制。如何配置,调优等。

 原创文章。欢迎转载,转载请标明出处 

转载于:https://www.cnblogs.com/wzzkaifa/p/7017132.html

本文来自互联网用户投稿,该文观点仅代表作者本人,不代表本站立场。本站仅提供信息存储空间服务,不拥有所有权,不承担相关法律责任。如若转载,请注明出处:http://www.mzph.cn/news/392432.shtml

如若内容造成侵权/违法违规/事实不符,请联系多彩编程网进行投诉反馈email:809451989@qq.com,一经查实,立即删除!

相关文章

leetcode 455. 分发饼干(贪心算法)

假设你是一位很棒的家长&#xff0c;想要给你的孩子们一些小饼干。但是&#xff0c;每个孩子最多只能给一块饼干。 对每个孩子 i&#xff0c;都有一个胃口值 g[i]&#xff0c;这是能让孩子们满足胃口的饼干的最小尺寸&#xff1b;并且每块饼干 j&#xff0c;都有一个尺寸 s[j]…

压缩/批量压缩/合并js文件

写在前面 如果文件少的话&#xff0c;直接去网站转化一下就行。 http://tool.oschina.net/jscompress?type3 1.压缩单个js文件 cnpm install uglify-js -g 安装 1>压缩单个js文件打开cmd,目录引到当前文件夹&#xff0c;cduglifyjs inet.js -o inet-min.js 或者 uglifyjs i…

angular依赖注入_Angular依赖注入简介

angular依赖注入by Neeraj Dana由Neeraj Dana In this article, we will see how the dependency injection of Angular works internally. Suppose we have a component named appcomponent which has a basic and simple structure as follows:在本文中&#xff0c;我们将看…

leetcode 85. 最大矩形(dp)

给定一个仅包含 0 和 1 、大小为 rows x cols 的二维二进制矩阵&#xff0c;找出只包含 1 的最大矩形&#xff0c;并返回其面积。 示例 1&#xff1a; 输入&#xff1a;matrix [[“1”,“0”,“1”,“0”,“0”],[“1”,“0”,“1”,“1”,“1”],[“1”,“1”,“1”,“1”,“…

如何查看系统版本

1. winR,输入cmd&#xff0c;确定&#xff0c;打开命令窗口&#xff0c;输入msinfo32&#xff0c;注意要在英文状态下输入&#xff0c;回车。然后在弹出的窗口中就可以看到系统的具体版本号了。 2.winR,输入cmd&#xff0c;确定&#xff0c;打开命令窗口&#xff0c;输入ver&am…

java activemq jmx_通过JMX 获取Activemq 队列信息

首先在 activemq.xml 中新增以下属性在broker 节点新增属性 useJmx"true"在managementContext 节点配置断开与访问服务iP配置成功后启动下面来看测试代码/*** Title: ActivemqTest.java* Package activemq* Description: TODO(用一句话描述该文件做什么)* author LYL…

风能matlab仿真_发现潜力:使用计算机视觉对可再生风能发电场的主要区域进行分类(第1部分)

风能matlab仿真Github Repo: https://github.com/codeamt/WindFarmSpotterGithub回购&#xff1a; https : //github.com/codeamt/WindFarmSpotter This is a series:这是一个系列&#xff1a; Part 1: A Brief Introduction on Leveraging Edge Devices and Embedded AI to …

【Leetcode_easy】821. Shortest Distance to a Character

problem 821. Shortest Distance to a Character 参考 1. Leetcode_easy_821. Shortest Distance to a Character; 完转载于:https://www.cnblogs.com/happyamyhope/p/11214805.html

tdd测试驱动开发课程介绍_测试驱动开发的实用介绍

tdd测试驱动开发课程介绍by Luca Piccinelli通过卢卡皮奇内利 测试驱动开发很难&#xff01; 这是不为人知的事实。 (Test Driven Development is hard! This is the untold truth about it.) These days you read a ton of articles about all the advantages of doing Test …

软件安装(JDK+MySQL+TOMCAT)

一&#xff0c;JDK安装 1&#xff0c;查看当前Linux系统是否已经安装了JDK 输入 rpm -qa | grep java 如果有&#xff1a; 卸载两个openJDK&#xff0c;输入rpm -e --nodeps 要卸载的软件 2&#xff0c;上传JDK到Linux 3&#xff0c;安装jdk运行需要的插件yum install gl…

leetcode 205. 同构字符串(hash)

给定两个字符串 s 和 t&#xff0c;判断它们是否是同构的。 如果 s 中的字符可以被替换得到 t &#xff0c;那么这两个字符串是同构的。 所有出现的字符都必须用另一个字符替换&#xff0c;同时保留字符的顺序。两个字符不能映射到同一个字符上&#xff0c;但字符可以映射自己…

Java core 包_feilong-core 让Java开发更简便的工具包

## 背景在JAVA开发过程中,经常看到小伙伴直接从网上copy一长段代码来使用,又或者写的代码很长很长很长...**痛点在于:*** 难以阅读* 难以维护* sonar扫描结果债务长* codereview 被小伙伴鄙视* ....feilong-core focus on J2SE,是[feilong platform](https://github.com/venusd…

TensorFlow 2.X中的动手NLP深度学习模型准备

简介&#xff1a;为什么我写这篇文章 (Intro: why I wrote this post) Many state-of-the-art results in NLP problems are achieved by using DL (deep learning), and probably you want to use deep learning style to solve NLP problems as well. While there are a lot …

静态代码块

静态代码块 静态代码块&#xff1a;定义在成员位置&#xff0c;使用static修饰的代码块{ }。位置&#xff1a;类中方法外。执行&#xff1a;随着类的加载而执行且执行一次&#xff0c;优先于main方法和构造方法的执行。格式&#xff1a;作用&#xff1a; 给类变量进行初始化赋值…

异步api_如何设计无服务器异步API

异步apiby Garrett Vargas通过Garrett Vargas 如何设计无服务器异步API (How To Design a Serverless Async API) I recently ran a workshop to teach developers how to create an Alexa skill. The workshop material centered around a project to return car rental sear…

C# 序列化与反序列化json

与合作伙伴讨论问题&#xff0c;说到的c与c#数据的转换调用&#xff0c;正好就说到了序列化与反序列化&#xff0c;同样也可用于不同语言间的调用&#xff0c;做了基础示例&#xff0c;作以下整理&#xff1a; 1 using System.Data;2 using System.Drawing;3 using System.Linq…

学java 的要点_零基础学Java,掌握Java的基础要点

对于程序员群体来说&#xff0c;了解一定的技巧会对学习专业技能更有帮助&#xff0c;也更有助于在自己的职业发展中处于有利地位&#xff0c;无限互联Java培训专家今天就为大家总结Java程序员入门时需要掌握的基础要点&#xff1a;掌握静态方法和属性静态方法和属性用于描述某…

实验人员考评指标_了解实验指标

实验人员考评指标In the first part of my series on experimental design Thinking About Experimental Design, we covered the foundations of an experiment: the goals, the conditions, and the metrics. In this post, we will move away from the initial experimental…

leetcode 188. 买卖股票的最佳时机 IV(dp)

给定一个整数数组 prices &#xff0c;它的第 i 个元素 prices[i] 是一支给定的股票在第 i 天的价格。 设计一个算法来计算你所能获取的最大利润。你最多可以完成 k 笔交易。 注意&#xff1a;你不能同时参与多笔交易&#xff08;你必须在再次购买前出售掉之前的股票&#xf…

kotlin编写后台_在Kotlin编写图书馆的提示

kotlin编写后台by Adam Arold亚当阿罗德(Adam Arold) 在Kotlin编写图书馆的提示 (Tips for Writing a Library in Kotlin) Writing a library in Kotlin seems easy but it can get tricky if you want to support multiple platforms. In this article we’ll explore ways f…