tomcat jdbc SlowQueryReport的实现解读

为什么80%的码农都做不了架构师?>>>   hot3.png

##序 tomcat提供了JdbcInterceptor可以用来监控jdbc的执行情况,默认提供了好几个现成的interceptor可以用,SlowQueryReport以及SlowQueryReportJmx就是其中的两个。

##JdbcInterceptor的基本原理

/*** Abstract class that is to be extended for implementations of interceptors.* Everytime an operation is called on the {@link java.sql.Connection} object the* {@link #invoke(Object, Method, Object[])} method on the interceptor will be called.* Interceptors are useful to change or improve behavior of the connection pool.<br>* Interceptors can receive a set of properties. Each sub class is responsible for parsing the properties during runtime when they* are needed or simply override the {@link #setProperties(Map)} method.* Properties arrive in a key-value pair of Strings as they were received through the configuration.* This method is called once per cached connection object when the object is first configured.** @version 1.0*/
public abstract class JdbcInterceptor implements InvocationHandler {/*** {@link java.sql.Connection#close()} method name*/public static final String CLOSE_VAL = "close";/*** {@link Object#toString()} method name*/public static final String TOSTRING_VAL = "toString";/*** {@link java.sql.Connection#isClosed()} method name*/public static final String ISCLOSED_VAL = "isClosed";/*** {@link javax.sql.PooledConnection#getConnection()} method name*/public static final String GETCONNECTION_VAL = "getConnection";/*** {@link java.sql.Wrapper#unwrap(Class)} method name*/public static final String UNWRAP_VAL = "unwrap";/*** {@link java.sql.Wrapper#isWrapperFor(Class)} method name*/public static final String ISWRAPPERFOR_VAL = "isWrapperFor";/*** {@link java.sql.Connection#isValid(int)} method name*/public static final String ISVALID_VAL = "isValid";/*** {@link java.lang.Object#equals(Object)}*/public static final String EQUALS_VAL = "equals";/*** {@link java.lang.Object#hashCode()}*/public static final String HASHCODE_VAL = "hashCode";/*** Properties for this interceptor.*/protected Map<String,InterceptorProperty> properties = null;/*** The next interceptor in the chain*/private volatile JdbcInterceptor next = null;/*** Property that decides how we do string comparison, default is to use* {@link String#equals(Object)}. If set to <code>false</code> then the* equality operator (==) is used.*/private boolean useEquals = true;/*** Public constructor for instantiation through reflection*/public JdbcInterceptor() {// NOOP}/*** Gets invoked each time an operation on {@link java.sql.Connection} is invoked.* {@inheritDoc}*/@Overridepublic Object invoke(Object proxy, Method method, Object[] args) throws Throwable {if (getNext()!=null) return getNext().invoke(proxy,method,args);else throw new NullPointerException();}/*** Returns the next interceptor in the chain* @return the next interceptor in the chain*/public JdbcInterceptor getNext() {return next;}/*** configures the next interceptor in the chain* @param next The next chain item*/public void setNext(JdbcInterceptor next) {this.next = next;}/*** Performs a string comparison, using references unless the useEquals property is set to true.* @param name1 The first name* @param name2 The second name* @return true if name1 is equal to name2 based on {@link #useEquals}*/public boolean compare(String name1, String name2) {if (isUseEquals()) {return name1.equals(name2);} else {return name1==name2;}}/*** Compares a method name (String) to a method (Method)* {@link #compare(String,String)}* Uses reference comparison unless the useEquals property is set to true* @param methodName The method name* @param method The method* @return <code>true</code> if the name matches*/public boolean compare(String methodName, Method method) {return compare(methodName, method.getName());}/*** Gets called each time the connection is borrowed from the pool* This means that if an interceptor holds a reference to the connection* the interceptor can be reused for another connection.* <br>* This method may be called with null as both arguments when we are closing down the connection.* @param parent - the connection pool owning the connection* @param con - the pooled connection*/public abstract void reset(ConnectionPool parent, PooledConnection con);/*** Called when {@link java.sql.Connection#close()} is called on the underlying connection.* This is to notify the interceptors, that the physical connection has been released.* Implementation of this method should be thought through with care, as no actions should trigger an exception.* @param parent - the connection pool that this connection belongs to* @param con    - the pooled connection that holds this connection* @param finalizing - if this connection is finalizing. True means that the pooled connection will not reconnect the underlying connection*/public void disconnected(ConnectionPool parent, PooledConnection con, boolean finalizing) {}/*** Returns the properties configured for this interceptor* @return the configured properties for this interceptor*/public Map<String,InterceptorProperty> getProperties() {return properties;}/*** Called during the creation of an interceptor* The properties can be set during the configuration of an interceptor* Override this method to perform type casts between string values and object properties* @param properties The properties*/public void setProperties(Map<String,InterceptorProperty> properties) {this.properties = properties;final String useEquals = "useEquals";InterceptorProperty p = properties.get(useEquals);if (p!=null) {setUseEquals(Boolean.parseBoolean(p.getValue()));}}/*** @return true if the compare method uses the Object.equals(Object) method*         false if comparison is done on a reference level*/public boolean isUseEquals() {return useEquals;}/*** Set to true if string comparisons (for the {@link #compare(String, Method)} and {@link #compare(String, String)} methods) should use the Object.equals(Object) method* The default is false* @param useEquals <code>true</code> if equals will be used for comparisons*/public void setUseEquals(boolean useEquals) {this.useEquals = useEquals;}/*** This method is invoked by a connection pool when the pool is closed.* Interceptor classes can override this method if they keep static* variables or other tracking means around.* <b>This method is only invoked on a single instance of the interceptor, and not on every instance created.</b>* @param pool - the pool that is being closed.*/public void poolClosed(ConnectionPool pool) {// NOOP}/*** This method is invoked by a connection pool when the pool is first started up, usually when the first connection is requested.* Interceptor classes can override this method if they keep static* variables or other tracking means around.* <b>This method is only invoked on a single instance of the interceptor, and not on every instance created.</b>* @param pool - the pool that is being closed.*/public void poolStarted(ConnectionPool pool) {// NOOP}}

可以看到它实现了InvocationHandler这个接口,也就是使用的是java内置的动态代理技术,主要是因为jdbc本身就是面向接口编程的,因而用java内置的动态代理是水到渠成的。

##ConnectionPool tomcat-jdbc-8.5.11-sources.jar!/org/apache/tomcat/jdbc/pool/ConnectionPool.java

/*** configures a pooled connection as a proxy.* This Proxy implements {@link java.sql.Connection} and {@link javax.sql.PooledConnection} interfaces.* All calls on {@link java.sql.Connection} methods will be propagated down to the actual JDBC connection except for the* {@link java.sql.Connection#close()} method.* @param con a {@link PooledConnection} to wrap in a Proxy* @return a {@link java.sql.Connection} object wrapping a pooled connection.* @throws SQLException if an interceptor can't be configured, if the proxy can't be instantiated*/protected Connection setupConnection(PooledConnection con) throws SQLException {//fetch previously cached interceptor proxy - one per connectionJdbcInterceptor handler = con.getHandler();if (handler==null) {//build the proxy handlerhandler = new ProxyConnection(this,con,getPoolProperties().isUseEquals());//set up the interceptor chainPoolProperties.InterceptorDefinition[] proxies = getPoolProperties().getJdbcInterceptorsAsArray();for (int i=proxies.length-1; i>=0; i--) {try {//create a new instanceJdbcInterceptor interceptor = proxies[i].getInterceptorClass().newInstance();//configure propertiesinterceptor.setProperties(proxies[i].getProperties());//setup the chaininterceptor.setNext(handler);//call resetinterceptor.reset(this, con);//configure the last one to be held by the connectionhandler = interceptor;}catch(Exception x) {SQLException sx = new SQLException("Unable to instantiate interceptor chain.");sx.initCause(x);throw sx;}}//cache handler for the next iterationcon.setHandler(handler);} else {JdbcInterceptor next = handler;//we have a cached handler, reset itwhile (next!=null) {next.reset(this, con);next = next.getNext();}}try {getProxyConstructor(con.getXAConnection() != null);//create the proxy//TODO possible optimization, keep track if this connection was returned properly, and don't generate a new facadeConnection connection = null;if (getPoolProperties().getUseDisposableConnectionFacade() ) {connection = (Connection)proxyClassConstructor.newInstance(new Object[] { new DisposableConnectionFacade(handler) });} else {connection = (Connection)proxyClassConstructor.newInstance(new Object[] {handler});}//return the connectionreturn connection;}catch (Exception x) {SQLException s = new SQLException();s.initCause(x);throw s;}}

这里判断有没有interceptor,有的话,则创建ProxyConnection,然后构造interceptor的链 ##ProxyConnection

public class ProxyConnection extends JdbcInterceptor {protected PooledConnection connection = null;protected ConnectionPool pool = null;public PooledConnection getConnection() {return connection;}public void setConnection(PooledConnection connection) {this.connection = connection;}public ConnectionPool getPool() {return pool;}public void setPool(ConnectionPool pool) {this.pool = pool;}protected ProxyConnection(ConnectionPool parent, PooledConnection con,boolean useEquals) {pool = parent;connection = con;setUseEquals(useEquals);}@Overridepublic void reset(ConnectionPool parent, PooledConnection con) {this.pool = parent;this.connection = con;}public boolean isWrapperFor(Class<?> iface) {if (iface == XAConnection.class && connection.getXAConnection()!=null) {return true;} else {return (iface.isInstance(connection.getConnection()));}}public Object unwrap(Class<?> iface) throws SQLException {if (iface == PooledConnection.class) {return connection;}else if (iface == XAConnection.class) {return connection.getXAConnection();} else if (isWrapperFor(iface)) {return connection.getConnection();} else {throw new SQLException("Not a wrapper of "+iface.getName());}}@Overridepublic Object invoke(Object proxy, Method method, Object[] args) throws Throwable {if (compare(ISCLOSED_VAL,method)) {return Boolean.valueOf(isClosed());}if (compare(CLOSE_VAL,method)) {if (connection==null) return null; //noop for already closed.PooledConnection poolc = this.connection;this.connection = null;pool.returnConnection(poolc);return null;} else if (compare(TOSTRING_VAL,method)) {return this.toString();} else if (compare(GETCONNECTION_VAL,method) && connection!=null) {return connection.getConnection();} else if (method.getDeclaringClass().equals(XAConnection.class)) {try {return method.invoke(connection.getXAConnection(),args);}catch (Throwable t) {if (t instanceof InvocationTargetException) {throw t.getCause() != null ? t.getCause() : t;} else {throw t;}}}if (isClosed()) throw new SQLException("Connection has already been closed.");if (compare(UNWRAP_VAL,method)) {return unwrap((Class<?>)args[0]);} else if (compare(ISWRAPPERFOR_VAL,method)) {return Boolean.valueOf(this.isWrapperFor((Class<?>)args[0]));}try {PooledConnection poolc = connection;if (poolc!=null) {return method.invoke(poolc.getConnection(),args);} else {throw new SQLException("Connection has already been closed.");}}catch (Throwable t) {if (t instanceof InvocationTargetException) {throw t.getCause() != null ? t.getCause() : t;} else {throw t;}}}public boolean isClosed() {return connection==null || connection.isDiscarded();}public PooledConnection getDelegateConnection() {return connection;}public ConnectionPool getParentPool() {return pool;}@Overridepublic String toString() {return "ProxyConnection["+(connection!=null?connection.toString():"null")+"]";}}

ProxyConnection本身就是JdbcInterceptor,包装了PooledConnection

##AbstractCreateStatementInterceptor 这个是JdbcInterceptor的一个比较重要的扩展,SlowQueryReport就是基于这个扩展的。这个定义了一些抽象方法供子类实现。

/*** Abstraction interceptor. This component intercepts all calls to create some type of SQL statement.* By extending this class, one can intercept queries and update statements by overriding the {@link #createStatement(Object, Method, Object[], Object, long)}* method.* @version 1.0*/
public abstract class  AbstractCreateStatementInterceptor extends JdbcInterceptor {protected static final String CREATE_STATEMENT      = "createStatement";protected static final int    CREATE_STATEMENT_IDX  = 0;protected static final String PREPARE_STATEMENT     = "prepareStatement";protected static final int    PREPARE_STATEMENT_IDX = 1;protected static final String PREPARE_CALL          = "prepareCall";protected static final int    PREPARE_CALL_IDX      = 2;protected static final String[] STATEMENT_TYPES = {CREATE_STATEMENT, PREPARE_STATEMENT, PREPARE_CALL};protected static final int    STATEMENT_TYPE_COUNT = STATEMENT_TYPES.length;protected static final String EXECUTE        = "execute";protected static final String EXECUTE_QUERY  = "executeQuery";protected static final String EXECUTE_UPDATE = "executeUpdate";protected static final String EXECUTE_BATCH  = "executeBatch";protected static final String[] EXECUTE_TYPES = {EXECUTE, EXECUTE_QUERY, EXECUTE_UPDATE, EXECUTE_BATCH};public  AbstractCreateStatementInterceptor() {super();}/*** {@inheritDoc}*/@Overridepublic Object invoke(Object proxy, Method method, Object[] args) throws Throwable {if (compare(CLOSE_VAL,method)) {closeInvoked();return super.invoke(proxy, method, args);} else {boolean process = false;process = isStatement(method, process);if (process) {long start = System.currentTimeMillis();Object statement = super.invoke(proxy,method,args);long delta = System.currentTimeMillis() - start;return createStatement(proxy,method,args,statement, delta);} else {return super.invoke(proxy,method,args);}}}/*** This method will be invoked after a successful statement creation. This method can choose to return a wrapper* around the statement or return the statement itself.* If this method returns a wrapper then it should return a wrapper object that implements one of the following interfaces.* {@link java.sql.Statement}, {@link java.sql.PreparedStatement} or {@link java.sql.CallableStatement}* @param proxy the actual proxy object* @param method the method that was called. It will be one of the methods defined in {@link #STATEMENT_TYPES}* @param args the arguments to the method* @param statement the statement that the underlying connection created* @param time Elapsed time* @return a {@link java.sql.Statement} object*/public abstract Object createStatement(Object proxy, Method method, Object[] args, Object statement, long time);/*** Method invoked when the operation {@link java.sql.Connection#close()} is invoked.*/public abstract void closeInvoked();/*** Returns true if the method that is being invoked matches one of the statement types.** @param method the method being invoked on the proxy* @param process boolean result used for recursion* @return returns true if the method name matched*/protected boolean isStatement(Method method, boolean process){return process(STATEMENT_TYPES, method, process);}/*** Returns true if the method that is being invoked matches one of the execute types.** @param method the method being invoked on the proxy* @param process boolean result used for recursion* @return returns true if the method name matched*/protected boolean isExecute(Method method, boolean process){return process(EXECUTE_TYPES, method, process);}/** Returns true if the method that is being invoked matches one of the method names passed in* @param names list of method names that we want to intercept* @param method the method being invoked on the proxy* @param process boolean result used for recursion* @return returns true if the method name matched*/protected boolean process(String[] names, Method method, boolean process) {final String name = method.getName();for (int i=0; (!process) && i<names.length; i++) {process = compare(names[i],name);}return process;}/*** no-op for this interceptor. no state is stored.*/@Overridepublic void reset(ConnectionPool parent, PooledConnection con) {// NOOP}
}

##AbstractQueryReport 主要实现了createStatement方法:

/*** Creates a statement interceptor to monitor query response times*/@Overridepublic Object createStatement(Object proxy, Method method, Object[] args, Object statement, long time) {try {Object result = null;String name = method.getName();String sql = null;Constructor<?> constructor = null;if (compare(CREATE_STATEMENT,name)) {//createStatementconstructor = getConstructor(CREATE_STATEMENT_IDX,Statement.class);}else if (compare(PREPARE_STATEMENT,name)) {//prepareStatementsql = (String)args[0];constructor = getConstructor(PREPARE_STATEMENT_IDX,PreparedStatement.class);if (sql!=null) {prepareStatement(sql, time);}}else if (compare(PREPARE_CALL,name)) {//prepareCallsql = (String)args[0];constructor = getConstructor(PREPARE_CALL_IDX,CallableStatement.class);prepareCall(sql,time);}else {//do nothing, might be a future unsupported method//so we better bail out and let the system continuereturn statement;}result = constructor.newInstance(new Object[] { new StatementProxy(statement,sql) });return result;}catch (Exception x) {log.warn("Unable to create statement proxy for slow query report.",x);}return statement;}

这里同样适用了jdk的动态代理,包装了statement

/*** Class to measure query execute time**/protected class StatementProxy implements InvocationHandler {protected boolean closed = false;protected Object delegate;protected final String query;public StatementProxy(Object parent, String query) {this.delegate = parent;this.query = query;}@Overridepublic Object invoke(Object proxy, Method method, Object[] args) throws Throwable {//get the name of the method for comparisonfinal String name = method.getName();//was close invoked?boolean close = compare(JdbcInterceptor.CLOSE_VAL,name);//allow close to be called multiple timesif (close && closed) return null;//are we calling isClosed?if (compare(JdbcInterceptor.ISCLOSED_VAL,name)) return Boolean.valueOf(closed);//if we are calling anything else, bail outif (closed) throw new SQLException("Statement closed.");boolean process = false;//check to see if we are about to execute a queryprocess = isExecute( method, process);//if we are executing, get the current timelong start = (process)?System.currentTimeMillis():0;Object result =  null;try {//execute the queryresult =  method.invoke(delegate,args);}catch (Throwable t) {reportFailedQuery(query,args,name,start,t);if (t instanceof InvocationTargetException&& t.getCause() != null) {throw t.getCause();} else {throw t;}}//measure the timelong delta = (process)?(System.currentTimeMillis()-start):Long.MIN_VALUE;//see if we meet the requirements to measureif (delta>threshold) {try {//report the slow queryreportSlowQuery(query, args, name, start, delta);}catch (Exception t) {if (log.isWarnEnabled()) log.warn("Unable to process slow query",t);}} else if (process) {reportQuery(query, args, name, start, delta);}//perform close cleanupif (close) {closed=true;delegate = null;}return result;}}

这里记录了sql的执行耗时,然后跟阈值对比,判断是否记录到slow query

转载于:https://my.oschina.net/go4it/blog/1113969

本文来自互联网用户投稿,该文观点仅代表作者本人,不代表本站立场。本站仅提供信息存储空间服务,不拥有所有权,不承担相关法律责任。如若转载,请注明出处:http://www.mzph.cn/news/456095.shtml

如若内容造成侵权/违法违规/事实不符,请联系多彩编程网进行投诉反馈email:809451989@qq.com,一经查实,立即删除!

相关文章

【机器学习】Bagging和Boosting的区别(面试准备)

Baggging 和Boosting都是模型融合的方法&#xff0c;可以将弱分类器融合之后形成一个强分类器&#xff0c;而且融合之后的效果会比最好的弱分类器更好。 Bagging: 先介绍Bagging方法&#xff1a; Bagging即套袋法&#xff0c;其算法过程如下&#xff1a; 从原始样本集中抽取训…

线性判别结合源码分析LDA原理

1. LDA的思想 LDA线性判别分析也是一种经典的降维方法&#xff0c;LDA是一种监督学习的降维技术&#xff0c;也就是说它的数据集的每个样本是有类别输出的。这点和PCA不同。PCA是不考虑样本类别输出的无监督降维技术。LDA的思想可以用一句话概括&#xff0c;就是“投影后类内方…

RIFF文件规范

RIFF文件规范Peter Lee 2007-10-02 摘要&#xff1a;RIFF全称为资源互换文件格式&#xff08;Resources Interchange File Format&#xff09;&#xff0c;RIFF文件是windows环境下大部分多媒体文件遵循的一种文件结构&#xff0c;常见的如WAV文件、AVI文件等。RIFF可以看成一种…

【机器学习】feature_importances_ 参数源码解析

在用sklearn的时候经常用到feature_importances_ 来做特征筛选&#xff0c;那这个属性到底是啥呢。 分析源码发现来源于每个base_estimator的决策树的 feature_importances_ 由此发现计算逻辑来源于cython文件&#xff0c;这个文件可以在其github上查看源代码 而在DecisionTree…

【机器学习】 关联规则Apriori和mlxtend——推荐算法

引入&#xff1a; 啤酒与尿布的故事 关联规律挖掘&#xff1a;从交易数据中发现&#xff1a;买了X 还会买Y 的规则 关联规律挖掘‘购物篮分析’Market Basket Analysis&#xff08;MBA&#xff09; 关联规律->应用于推荐系统 1. 关联规则代码演示 使用的是mlxtend.frequent…

python中的内部函数和闭包函数

内部函数 在函数内部声明的函数就是内部函数。 格式&#xff1a; def 函数名():局部变量...def 内部函数名():Python功能代码...示例&#xff1a; def funName():name dragon#定义一个内部函数def inner():print(我是内部函数)‘’‘注释&#xff1a; 1.内部函数的本质就是局…

【机器学习】K-Means(非监督学习)学习及实例使用其将图片压缩

非监督学习之k-means K-means通常被称为劳埃德算法&#xff0c;这在数据聚类中是最经典的&#xff0c;也是相对容易理解的模型。算法执行的过程分为4个阶段。 1、从数据中选择k个对象作为初始聚类中心; 2、计算每个聚类对象到聚类中心的距离来划分&#xff1b; 3、再次计算…

CloudCC CRM:物联网必将成为CRM的推动力

CRM热门话题背后的主要推动力包括云、社交、移动和大数据&#xff0c;CloudCC CRM认为物联网必将成为CRM的推动力&#xff0c;也就是传感器将事物连接到互联网&#xff0c;创建之前我们从未想到的新型服务。 社交&#xff1a;在销售、市场和客户服务部门&#xff0c;营销人员正…

【tensorflow】tensorflow -gpu安装及jupyter环境更改

tensorflow -gpu安装 首先&#xff0c;安装Anoconda 1. 官网下载点我&#xff1a;2.安装 点击 python 3.6 version自动下载x64版&#xff0c;下载好之后&#xff0c;然后安装。 如图&#xff0c;打上勾之后&#xff0c;一路next3.打开终端 1&#xff09;输入conda –-version …

【机器学习】朴素贝叶斯介绍及实例--对短信进行二分类 使用多项式分布

贝叶斯 首先什么是贝叶斯&#xff1f; 一个例子&#xff0c;现分别有 A、B 两个容器&#xff0c;在容器 A 里分别有 7 个红球和 3 个白球&#xff0c;在容器 B 里有 1 个红球和 9 个白球&#xff0c;现已知从这两个容器里任意抽出了一个球&#xff0c;且是红球&#xff0c;问这…

Gartner分享物联网和智慧城市最新数据

主题为“移我所想 Mobile is me”的2016世界移动大会上海展正在上海如火如荼的举行&#xff0c;Gartner也在第一时间分享了最新的市场数据&#xff0c;包括企业级用户最为关注的物联网和智慧城市的调查预测报告&#xff0c;下面就一起来看看吧&#xff01; 智慧城市与物联网 物…

作业帮电脑版在线使用_作业帮:创新科技驱动在线教育新模式

10月15日&#xff0c;在线教育领军企业作业帮在中国校长大会在线教育论坛上&#xff0c;独家发布《学习的真相&#xff1a;全国K12学情大数据及学习洞察》&#xff0c;宣布已推出作业帮直播课“名师大招”课程体系&#xff0c;集中展示多款面向K12人群的教育黑科技和硬件产品。…

【机器学习】DBSCAN聚类算法—优于Kmean—(理论+图解+python代码)

一、前言 二、DBSCAN聚类算法 三、参数选择 四、DBSCAN算法迭代可视化展示 五、常用的评估方法&#xff1a;轮廓系数 六、用Python实现DBSCAN聚类算法 一、前言 去年学聚类算法的R语言的时候&#xff0c;有层次聚类、系统聚类、K-means聚类、K中心聚类&#xff0c;最后呢…

卡巴斯基:风险无国界 网络安全从业者要与小网民保持一致

“互联网没有国界&#xff0c;每个国家碰到的问题基本上是类似的。对于网络犯罪这件事&#xff0c;并不针对哪个国家&#xff0c;任何有弱点、有机会的地方&#xff0c;黑客都会去。”卡巴斯基公司CEO尤金卡巴斯基在接受未来网&#xff08;微信公众号lovek618&#xff09;记者采…

windows无法访问指定设备路径或文件_完全免费的文件数据恢复工具:Microsoft Windows File Recovery...

意外删除文件的经历是大多数人都遇到过&#xff0c;但是幸运的是有一种“后悔药”可以吃&#xff0c;因为我们可以使用一些【数据恢复软件】以找回一些已删除的文件。市面上有很多这类型的软件&#xff0c;例如EasyRecovery、DiskGenius、Recuva 等软件。但是&#xff0c;功能强…

【机器学习】XGBoost集成算法——(理论+图解+python代码比较其他算法使用天池蒸汽数据)

一、集成算法思想 二、XGBoost基本思想 三、用python实现XGBoost算法 在竞赛题中经常会用到XGBoost算法&#xff0c;用这个算法通常会使我们模型的准确率有一个较大的提升。既然它效果这么好&#xff0c;那么它从头到尾做了一件什么事呢&#xff1f;以及它是怎么样去做的呢&a…

1万条数据大概占多大空间_国漫丨2019年上半年漫画数据报告

文 │ 骨朵国漫一、各漫画平台总体趋势1、快看漫画快看漫画平台2019年Q2各月评论数较Q1有较明显的下滑&#xff0c;月评论数都在400万条左右&#xff1b;收藏数方面&#xff0c;2019年2月达到了半年内最高值&#xff0c;为2660.1万条。2、看漫画、知音漫客等平台(小明太极旗下)…

【机器学习】 LightGBM——优于XGBoost提升的集成算法(安装+对比XGBoost改进+参数说明+python代码实例)

中文官方文档&#xff1a;http://lightgbm.apachecn.org/cn/latest/Installation-Guide.html 英文官方文档&#xff1a;https://lightgbm.readthedocs.io/en/latest/ 一、lightGBM安装 在anaconda中输入&#xff1a;pip install lightGBM即可 输入import lightgbm as lgb做测…

轻量级代码生成器-OnlyCoder 第一篇

程序猿利器&#xff1a;代码生成器&#xff0c;使用代码生成器已经好几年了&#xff0c;增删改查各种生成&#xff0c;从UI到DATA层均生成过。之前有使用过动软的&#xff0c;T4模板等.... T4生成实体还是没有问题的&#xff0c;但是生成MVC视图就有点烦杂了&#xff0c;动软给…

从Java到Go面向对象--继承思想.md

2019独角兽企业重金招聘Python工程师标准>>> Go语言实现继承 我们都知道Java中通过extends来实现对象的继承&#xff0c;那么Go中怎么实现继承呢&#xff1f;Go语言没有继承&#xff0c;只有组合。我们知道&#xff0c;继承的目的是什么&#xff0c;是类的重用&…