背景:
工作需要经常使用到weblogic中间件产品,在维护过程中有遇见的一些常见故障问题,这里分享给大家。
问题一:密码文件报错
问题描述:
weblogic应用在启动过程中出现如下的报错内容:
# tail -f nohup.out
at weblogic.security.providers.authentication.LDAPAtnLoginModuleImpl.login(LDAPAtnLoginModuleImpl.java:261)
at com.bea.common.security.internal.service.LoginModuleWrapper$1.run(LoginModuleWrapper.java:110)
at java.security.AccessController.doPrivileged(Native Method)
at com.bea.common.security.internal.service.LoginModuleWrapper.login(LoginModuleWrapper.java:106)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
Truncated. see log file for complete stacktrace
>
<Dec 31, 2018 5:20:44 PM CST> <Notice> <WebLogicServer> <BEA-000365> <Server state changed to FAILED>
<Dec 31, 2018 5:20:44 PM CST> <Error> <WebLogicServer> <BEA-000383> <A critical service failed. The server will shut itself down>
<Dec 31, 2018 5:20:44 PM CST> <Notice> <WebLogicServer> <BEA-000365> <Server state changed to FORCE_SHUTTING_DOWN>
问题分析:
显示如上的错误,是weblogic登入的密码文件出错了,
问题解决:
解决思路:只要是修改一下密码文件就可以了
进入域的根目录文件下
$ cd /weblogic/user_projects/domains/pubdomain01/servers/pubserver01/security
$ vim boot.properties
# Generated by Configuration Wizard on Tue Apr 24 10:55:31 CST 2018
username= weblogic ---------创建时的weblogic控制台的用户,
password=servyou2014 ----------创建时的weblogic控制台用户的密码,
$ cd /weblogic/user_projects/domains/sbfpubdomain01/bin
$ nohup ./startWebLogic.sh & --------执行启动
问题二:有用过root启动过weblogic服务,文件权限不一致了
问题描述:
启动weblogic的使用报错如下:
$ tail -f nohup.out
java.lang.ClassCastException: com.octetstring.vde.backend.BackendRoot cannot be cast to com.octetstring.vde.backend.standard.BackendStandardat weblogic.ldap.EmbeddedLDAP.start(EmbeddedLDAP.java:303)at weblogic.t3.srvr.SubsystemRequest.run(SubsystemRequest.java:64)at weblogic.work.ExecuteThread.execute(ExecuteThread.java:256)at weblogic.work.ExecuteThread.run(ExecuteThread.java:221)>
<Jan 1, 2019 7:48:34 PM CST> <Notice> <WebLogicServer> <BEA-000365> <Server state changed to FAILED>
<Jan 1, 2019 7:48:34 PM CST> <Error> <WebLogicServer> <BEA-000383> <A critical service failed. The server will shut itself down>
<Jan 1, 2019 7:48:34 PM CST> <Notice> <WebLogicServer> <BEA-000365> <Server state changed to FORCE_SHUTTING_DOWN>
问题分析:
报错原因:说是因为有用过root启动过weblogic服务,文件权限不一致了。
问题解决:
解决办法:将weblogic目录下的所有的子目录的所有者和所属组修改为weblogic用户就可以了。
在Weblogic用户目录下的weblogic路径下,执行下面的命令:
# chown -R weblogic:weblogic /weblogic
重新启动weblogic就可以了
问题三:调用其他服务依赖失败
问题描述:
业务办理报错如下:
[ERROR][2019/11/11 08:33:07.658][servyou.tyjrpt.common.logback.rpc.RpcLogbackAspect][org.springframework.jms.listener][sequenceid:36CF6D8519C64AEA9F44943D0C5FF0F7]调用社保核心[未知]服务 失败!响应时间:19,请求报文:<?xml version="1.0" encoding="UTF-8" standalone="yes"?>,异常:[Security:090398]Invalid Subject: principals=[weblogic, Administrators]
[ERROR][2019/11/11 08:33:07.658][BaseLogAspect][org.springframework.jms.listener][sequenceid:36CF6D8519C64AEA9F44943D0C5FF0F7]调用:HdxxService.hdxxxz(..):失败! [timecost:22] 异常为SecurityException:[Security:090398]Invalid Subject: principals=[weblogic, Administrators]
[ERROR][2019/11/11 08:33:07.658][BaseLogAspect][org.springframework.jms.listener][sequenceid:36CF6D8519C64AEA9F44943D0C5FF0F7]调用:HdxxFacade.hdxxxz(..):失败! [timecost:22] 异常为SecurityException:[Security:090398]Invalid Subject: principals=[weblogic, Administrators]
[ERROR][2019/11/11 08:33:07.658][cn.com.servyou.tyjrpt.xj.sb.jms.DwhzhdxxJmsMDB][org.springframework.jms.listener][sequenceid:36CF6D8519C64AEA9F44943D0C5FF0F7]核定信息下载失败java.lang.SecurityException: [Security:090398]Invalid Subject: principals=[weblogic, Administrators]at weblogic.security.service.SecurityServiceManager.seal(SecurityServiceManager.java:848)at weblogic.security.service.SecurityServiceManager.getSealedSubjectFromWire(SecurityServiceManager.java:522)at weblogic.rjvm.MsgAbbrevInputStream.getSubject(MsgAbbrevInputStream.java:357)
Wrapped by: java.lang.SecurityException: [Security:090398]Invalid Subject: principals=[weblogic, Administrators]at weblogic.rjvm.ResponseImpl.unmarshalReturn(ResponseImpl.java:237)at weblogic.rmi.cluster.ClusterableRemoteRef.invoke(ClusterableRemoteRef.java:350)at weblogic.rmi.cluster.ClusterableRemoteRef.invoke(ClusterableRemoteRef.java:261)
问题分析:
注:这里根据报错的信息可以确定是调用核心服务出现异常,异常是直接调用不到核心服务,
出现如上的报错,有以下几种原因:
1、核心服务异常,没有启动或者是启动异常导致;
2、该服务异常,或者是服务启动异常;
3、是该服务和核心服务之间的网络策略被改动是拒绝状态;根据大致的这几种情况去排查,这里的问题是该服务启动有异常,其他两种都排查掉,没有问题。
问题解决:
这里时候可以检查一下启动日志就知道了
$ nohup.out
<Nov 11, 2019 8:10:11 PM CST> <Notice> <WebLogicServer> <BEA-000365> <Server state changed to STANDBY>
<Nov 11, 2019 8:10:11 PM CST> <Notice> <WebLogicServer> <BEA-000365> <Server state changed to STARTING>
<Nov 11, 2019 8:10:18 PM CST> <Notice> <Log Management> <BEA-170027> <The Server has established connection with the Domain level Diagnostic Service successfully.>
<Nov 11, 2019 8:10:18 PM CST> <Notice> <Cluster> <BEA-000197> <Listening for announcements from cluster using unicast cluster messaging>
<Nov 11, 2019 8:10:18 PM CST> <Notice> <Cluster> <BEA-000133> <Waiting to synchronize with other running members of cluster_sbfjs.>
<Nov 11, 2019 8:10:48 PM CST> <Notice> <WebLogicServer> <BEA-000365> <Server state changed to ADMIN>
<Nov 11, 2019 8:10:48 PM CST> <Notice> <WebLogicServer> <BEA-000365> <Server state changed to RESUMING>
<Nov 11, 2019 8:10:48 PM CST> <Notice> <Cluster> <BEA-000162> <Starting "async" replication service with remote cluster address "null">
<Nov 11, 2019 8:10:48 PM CST> <Notice> <Server> <BEA-002613> <Channel "Default" is now listening on 10.104.102.113:7201 for protocols iiop, t3, CLUSTER-BROADCAST, ldap, snmp, http.>
<Nov 11, 2019 8:10:48 PM CST> <Notice> <WebLogicServer> <BEA-000330> <Started WebLogic Managed Server "sbfjsserver01" for domain
注:根据上面的启动日志发现,weblogic根本就没有识别部署的war程序,导致整个业务办理流程出现错误,
解决思路:
需要先停掉weblogic服务,然后需要重新部署该war包应用,然后再重新启动weblogic应用服务就可以了。
启动weblogic服务之前,需要先将weblogic应用相关的缓存数据清除掉,不然会影响该war应用的。