sparksql一些指标

统计指标

select
substr(tb.begin_address_code , 1 ,4) as begin_address_code ,
count(distinct vehicle_license) as dayVehicleCount
from
(select
begin_address_code ,
vehicle_license
from
order
where
date_format(create_time , 'yyyy-MM-dd')  = '2020-02-15
') tb
group by
substr(tb.begin_address_code , 1 ,4)

汇总指标

select
monthVehicle.begin_address_code ,
NVL(monthVehicle.monthVehicleCount ,0) as monthVehicleCount,
NVL(dayVehicle.dayVehicleCount , 0) as dayVehicleCount
from
monthVehicle left join dayVehicle
on
monthVehicle.begin_address_code = dayVehicle.begin_address_code

日指标

select
substr(tb.begin_address_code , 1 , 4)  as begin_address_code  ,
count(1) as dayOrderCount
from
(select begin_address_code from order
where
date_format(create_time , 'yyyy-MM-dd') = '2020-02-15') tb
group by
substr(tb.begin_address_code , 1 , 4)

周指标

select
substr(tb.begin_address_code , 1 , 4)  as begin_address_code  ,
count(1) as weekOrderCount
from
(select begin_address_code from order
where
weekofyear(create_time) = '29') tb
group by
substr(tb.begin_address_code , 1 , 4)

月指标

select
substr(tb.begin_address_code , 1 , 4)  as begin_address_code  ,
count(1) as monthOrderCount
from
(select begin_address_code from order
where
date_format(create_time , 'yyyy-MM') = '2020-02') tb
group by
substr(tb.begin_address_code , 1 , 4)

汇总

select
tb1.begin_address_code ,
tb1.monthOrderCount ,
tb1.weekOrderCount ,
dayOrder.dayOrderCount
from
(select
monthOrder.begin_address_code ,
monthOrder.monthOrderCount ,
weekOrder.weekOrderCount
from
monthOrder left join weekOrder
on
monthOrder.begin_address_code  =  weekOrder.begin_address_code)  tb1 left join dayOrder
on
tb1.begin_address_code = dayOrder.begin_address_code

汇总

select
cast(
concat(if(v_begin_address_code is null ,o_begin_address_code , v_begin_address_code)
, '00')
as string) as rk,
cast(if(monthVehicleCount is null , 0 ,monthVehicleCount) as string) as monthVehicleCount ,
cast(if(dayVehicleCount is null , 0 , dayVehicleCount) as string) as dayVehicleCount ,
cast(
concat(if(o_begin_address_code is null ,v_begin_address_code , o_begin_address_code)
, '00')
as string) as o_begin_address_code,
cast(if(monthOrderCount is null , 0 , monthOrderCount) as string) as monthOrderCount ,
cast(if(weekOrderCount is null, 0 , weekOrderCount) as string) as weekOrderCount ,
cast(if(dayOrderCount is null ,0, dayOrderCount) as string) as dayOrderCount
from
(select
vehcileCount.begin_address_code as v_begin_address_code,
vehcileCount.monthVehicleCount ,
vehcileCount.dayVehicleCount ,
_totalOrder.begin_address_code as o_begin_address_code ,
_totalOrder.monthOrderCount ,
_totalOrder.weekOrderCount ,
_totalOrder.dayOrderCount
from
vehcileCount full outer join _totalOrder
on
vehcileCount.begin_address_code = _totalOrder.begin_address_code
) tb

本文来自互联网用户投稿,该文观点仅代表作者本人,不代表本站立场。本站仅提供信息存储空间服务,不拥有所有权,不承担相关法律责任。如若转载,请注明出处:http://www.mzph.cn/news/508767.shtml

如若内容造成侵权/违法违规/事实不符,请联系多彩编程网进行投诉反馈email:809451989@qq.com,一经查实,立即删除!

相关文章

sparkConf常见参数设置

def getSparkConf():SparkConf {val sparkConf: SparkConf new SparkConf().set("spark.driver.cores","4") //设置driver的CPU核数.set("spark.driver.maxResultSize","2g") //设置driver端结果存放的最大容量,这里设置…

sparkSession常见参数设置

def getSparkSession(sparkConf:SparkConf):SparkSession {val sparkSession: SparkSession SparkSession.builder().config(sparkConf)//调度模式.config("spark.scheduler.mode", "FAIR").config("spark.executor.memoryOverhead", "51…

关于kafka中acks是否可以为all

kafka源码中有这样一段代码: org.apache.kafka.clients.producer.KafkaProducer private static int parseAcks(String acksString) {try {return acksString.trim().equalsIgnoreCase("all") ? -1 : Integer.parseInt(acksString.trim());} catch (Numb…

关于统计时间切片标签的一些sql

------当天付费明细表 DROP TABLE IF EXISTS rpt.tmp_mm_rb_daily_ffmx; create table rpt.tmp_mm_rb_daily_ffmx as select a.* FROM (select c.feemsisdn, c.destmsisdn, c.day, c.price/1000 fee, c.contentid, dc.content_name, c.ordernumber, c.cdrtime, c.createtime, c…

hadoop 二次开发DatanodeWriteTimeout设置

int getDatanodeWriteTimeout(int numNodes) {return this.dfsClientConf.confTime > 0 ? this.dfsClientConf.confTime 5000 * numNodes : 0;}int getDatanodeReadTimeout(int numNodes) {return this.dfsClientConf.socketTimeout > 0 ? 5000 * numNodes this.dfsC…

聚类算法

假定样本集 D {X1, 的,…, Xm} 包含 m 个无标记样本, 每个样本 X (X1; X2;… ; Xn) 是一个 n 维特征向量,则聚类算法将样本 集 D 划分为 k 个不相交的簇 {Gl I l 1, 2;… ,时,其中…

k-means均值向量

给定样本集 D {Xl) 的,… ,xm}, “k 均值” (k-means )算法针对聚类所 得簇划分 C {C1, C2,…, Ck} 最小化平方误差 ι ELL Ilx 一队IIL il EGi 其中队甘il LEGi X 是簇 q 的均值向量.在一定程度上 刻画了簇内样本围绕簇均值向量的紧密程度, E 值越小则…

学习向量量化

与 k 均值算法类似,“学习向量量化” (Learning Vector Quantization,简 称 LVQ)也是试图找到一组原型向量来刻画聚类结构, 但与一般聚类算法不同 的是, LVQ 假设数据样本带有类别标记,学习过程利用样本的这些监督信息…

k 近邻加权平均

k 近邻(k-Nearest Neighbor,简称 kNN)学习是一种常用的监督学习方法, 其工作机制非常简单: 给定测试样本?基于某种距离度量找出训练集中与其最 靠近的 k 个训练样本,然后基于这 k 个"邻居"的信息来进行预测. 通常, 在分…

k 近邻降维

k 近邻(k-Nearest Neighbor,简称 kNN)学习是一种常用的监督学习方法, 其工作机制非常简单: 给定测试样本?基于某种距离度量找出训练集中与其最 靠近的 k 个训练样本,然后基于这 k 个"邻居"的信息来进行预测. 通常, 在分…

维度建模工具

幵始维度建模工作前,项目组需要理解业务需求,以及作为基础的源数据的实际情况。 通过与、 Ik务代表交流来发现需求,用于理解他们的基于关键性能指标、竞争性商业问题、 决策制定过程、支持分析需求的目标。同时,数据实际情况可以通…

Cube和Grouping 和Rollup

增强的聚合 Cube和Grouping 和Rollup 这几个分析函数通常用于OLAP中,不能累加,而且需要根据不同维度上钻和下钻的指标统计,比如,分小时、天、月的UV数。 GROUPING SETS 在一个GROUP BY查询中,根据不同的维度组合进行聚…

常见维度建模错误

需要避免的常见维度建模错误 错误 10: 在事实表中放入文本属性 要从数据仓库事实表中 挑出这些文本属性,并将它们放入维度表中。 错误 9: 限制使用冗长的描述符以节省空间 维度表从几何上看总是比事实表小很多。 错误 8: 将层次划分为多个维度 以用户看来最自然最 有…

2020-09-21

columns has 234 elements while hbase.columns.mapping has 92 elements (counting the key if implicit)) 根本原因: 对于4000个字符,hive Metastore中SERDE_PARAMS表中PARAM_VALUE字段的字符限制是此问题的根本原因。 此限制可防止Hive创建具有高列数…

2020-09-23

insert into table ads_user_action_convert_day select ‘2019-02-10’, uv.day_count, ua.order_count, cast(ua.order_count/uv.day_count as decimal(10,2)) visitor2order_convert_ratio, ua.payment_count, cast(ua.payment_count/ua.order_count as decimal(10,2)) orde…

2020-09-28

Mybatis中#{}和${} 1、#{}将传入的数据都当成一个字符串,会对自动传入的数据加一个引号(单引号?双引号?加了引号就对了)如: //传入的值是sex order by #{column} 解析后为 order by “sex” //将会出错 2…

PageUtil

Data NoArgsConstructor public class PageUtil implements Serializable {private static final long serialVersionUID 1L;/*** 总记录数*/private int totalCount;/*** 每页记录数*/private int pageSize;/*** 总页数*/private int totalPage;/*** 当前页数*/private int c…

springboot+事务,多张表的操作事务回滚

第一步,在springboot的启动类上开启事务,注解 EnableTransactionManagement 第二步:事务注解,回滚 Transactional(rollbackFor Exception.class) //Transactional和try catch捕获异常会让注解失效,所以: 第…

@EnableTransactionManagement

GetMapping("/saveData")ApiOperation("传感器数据上云-红外测温及气体")Transactional(rollbackFor Exception.class)public WebResult saveData(InfrareDataReqVo infrareDataReqVo){try {PigHeatEntity pigHeatEntitynew PigHeatEntity();pigHeatEntit…

根据经纬度显示地图轨迹

后端代码 public boolean saveCarGPSData(CarGPSReqVo carGPSReqVo) {Map<String,String> paramsnew HashMap<>(2);params.put("key",trsConfig.getCarGPS_key());params.put("coorType",carGPSReqVo.getCoorType());params.put("pageNu…