从单体架构到微服务架构的演变, 一个业务请求往往会流转多个服务, 大型互联网产品服务架构尤为复杂,腾讯的抢红包服务, 阿里的交易支付服务, 可能就流转成百上千个服务节点, 面对众多服务, 如何监控管理?服务请求一旦出现问题, 如何快速定位问题?如何保障服务的高可用, 做到全面的监控与预警?如何分析统计服务的运行状况?看下链路监控产品如何解决这些问题。
优势:
报表名称 | 报表用途 |
---|---|
Transaction实时报表 | 一段代码的运行时间/次数/分布、比如URL/Cache/SQL执行次数和响应时间 |
Event实时报表 | 事件产生的次数/分布,比如出现一个异常 |
Problem实时报表 | 根据Transaction/Event数据分析出来的系统出现的异常,包括访问较慢的程序等 |
Heartbeat实时报表 | JVM内部一些状态信息,Load/Memory/GC/Thread等 |
Metric实时报表 | 业务指标采集监控报表 |
Matrix实时报表 | 一个请求调用分布统计(一次请求中调用多少次SQL/RPC/Cache等),可评估应用设计的合理性 |
... | ... |
整体设计 简单即是最好原则设计, 主要分为三个模块cat-client,cat-consumer,cat-home。
客户端设计
客户端设计是CAT系统设计中最为核心的一个环节,客户端要求是做到API简单、高可靠性能、无论在任何场景下客户端都不能影响各业务服务的性能(监控只是公司核心业务流程一个旁路环节)。
服务端设计 服务端单机cat-consumer的整体架构:
当某个报表处理器处理来不及时候,比如Transaction报表处理比较慢,可以通过配置支持开启多个Transaction处理线程,并发消费消息。
PS:不建议在Windows下部署, 设计上对window支持不好, 容易出各种问题。
下载CAT源码, 如GIT方式过慢, 可用Download Zip 方式打包下载
构建CAT服务war包 可以导入IDEA工程进行编译, 或者直接用MAVEN进行编译,这里编译的目录是:cat-home
将MAVEN加入到系统PATH, 执行mvn命令:
mvn clean install -Dmaven.test.skip=true
创建数据库
先创建CAT数据库, 采用utf8mb4字符集, 再导入{CAT_SRC}/script/目录下的CatApplication.sql脚本。
创建目录,因为cat需要/data的全部权限,运行盘下的/data/appdatas/cat和/data/applogs/cat有读写权限
mkdir /data/appdatas/cat/
chmod -R 777 /data/appdatas/cat/
将打包好的war包传入tomcat的webapp下
创建客户端的配置/data/appdatas/cat/client.xml (客户端使用)
<?xml version="1.0" encoding="utf-8"?>
<config mode="client">
<servers>
<server ip="127.0.0.1" port="2280" http-port="8080"/>
</servers>
</config>
创建服务端的配置/data/appdatas/cat/datasources.xml (服务端使用)
<?xml version="1.0" encoding="utf-8"?>
<data-sources>
<data-source id="cat">
<maximum-pool-size>3</maximum-pool-size>
<connection-timeout>1s</connection-timeout>
<idle-timeout>10m</idle-timeout>
<statement-cache-size>1000</statement-cache-size>
<properties>
<driver>com.mysql.jdbc.Driver</driver>
<url><![CDATA[jdbc:mysql://127.0.0.1:3306/cat]]></url> <!-- 请替换为真实数据库URL及Port -->
<user>root</user> <!-- 请替换为真实数据库用户名 -->
<password>123456</password> <!-- 请替换为真实数据库密码 -->
<connectionProperties><![CDATA[useUnicode=true&characterEncoding=UTF-8&autoReconnect=true&socketTimeout=120000]]></connectionProperties>
</properties>
</data-source>
</data-sources>
Tomcat配置,修改tomcat conf 目录下 server.xml, 检查好端口没有被其他程序占用。
<Connector port="8080" protocol="HTTP/1.1"
URIEncoding="utf-8" connectionTimeout="20000"
redirectPort="8443" /> <!-- 增加 URIEncoding="utf-8" -->
如需内存不足,需作调整
CATALINA_OPTS="-Xms1024m -Xmx2048m -Xss1024K -XX:MetaspaceSize=512m -XX:MaxMetaspaceSize=1024m"
启动
进入tomcat bin目录下
bash startup.sh
访问cat客户端
具体可参考官方文档: CAT集群部署
设计四个服务:网关服务、订单服务、账户服务和库存服务, 三层调用关系监控
命名为
cat-demo
pom依赖
<dependencies>
<!-- spring boot 依赖 -->
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<!-- Nacos服务注册发现依赖 -->
<dependency>
<groupId>com.alibaba.cloud</groupId>
<artifactId>spring-cloud-starter-alibaba-nacos-discovery</artifactId>
</dependency>
<!-- Spring Boot 监控组件依赖 -->
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-actuator</artifactId>
</dependency>
<!-- CAT 组件依赖-->
<dependency>
<groupId>com.dianping.cat</groupId>
<artifactId>cat-client</artifactId>
<version>3.0.1</version>
</dependency>
</dependencies>
父级为
cat-demo
,命名为cat-gateway
启动类
@SpringBootApplication
@EnableDiscoveryClient
@ComponentScan(basePackages = {"com.xiaobear"})
@RestController
public class CatGateWayApplication {
public static void main(String[] args) {
SpringApplication.run(CatGateWayApplication.class, args);
}
@Resource
private RestTemplate restTemplate;
/**
* 指向订单服务的接口
*/
@Value("${service2.address:localhost:8082}")
private String serviceAddress;
/**
* 模拟正常的请求
* @return
* @throws Exception
*/
@RequestMapping("/gateway")
public String gateway() throws Exception {
Thread.sleep(100);
String response = restTemplate.getForObject("http://" + serviceAddress + "/order", String.class);
return "gateway response ==> " + response;
}
/**
* 模拟一个请求异常
* @return
* @throws Exception
*/
@RequestMapping("/timeout")
public String timeout() throws Exception {
Transaction t = Cat.newTransaction(CatConstants.TYPE_URL, "timeout");
try {
Thread.sleep(100);
String response = restTemplate.getForObject("http://" + serviceAddress + "/timeout", String.class);
return response;
}catch(Exception e) {
Cat.getProducer().logError(e);
t.setStatus(e);
throw e;
}finally {
t.complete();
}
}
@Bean
RestTemplate restTemplate() {
RestTemplate restTemplate = new RestTemplate();
// 保存和传递调用链上下文
restTemplate.setInterceptors(Collections.singletonList(new CatRestInterceptor()));
return restTemplate;
}
}
yml配置文件
server:
port: 8081
spring:
application:
name: cat-gateway
cloud:
nacos:
discovery:
ip: 127.0.0.1:8848
#暴露端口
management:
endpoint:
web:
exposure:
include: ‘*’
工具类
CatContext
: 存放调用链上下文信息,这里需要注意的是cat的依赖包为3.0的,新版本4的没有一些接口Cat.Context
public
class
CatContext
implements
Cat.Context {
private
Map<String, String> properties = new
HashMap<>();
@Override
public
void
addProperty(String s, String s1) {
properties.put(s, s1);
}
@Override
public
String
getProperty(String s) {
return properties.get(s);
}
}
CatServletFilter
:过滤器,过滤访问的一些路径
public class CatServletFilter implements Filter {
private String[]
urlPatterns = new String[0];
/**
*过滤初始化
* @param filterConfig
* @throws ServletException
*/
@Override
public void init(FilterConfig filterConfig) throws ServletException {
String parameter = filterConfig.getInitParameter("CatHttpModuleUrlPatterns");
if (null != parameter){
parameter = parameter.trim();
urlPatterns = parameter.split(",");
for (int i = 0; i < urlPatterns.length; i++) {
urlPatterns[i] = urlPatterns[i].trim();
}
}
}
/**
* 请求过滤处理
* @param servletRequest
* @param servletResponse
* @param filterChain
* @throws IOException
* @throws ServletException
*/
@Override
public void doFilter(ServletRequest servletRequest, ServletResponse servletResponse, FilterChain filterChain) throws IOException, ServletException {
HttpServletRequest request = (HttpServletRequest) servletRequest;
String url = request.getRequestURL().toString();
for (String urlPattern : urlPatterns) {
if (url.startsWith(urlPattern)) {
url = urlPattern;
}
}
//存放消息的上下文对象
CatContext catContext = new CatContext();
catContext.addProperty(Cat.Context.CHILD, request.getHeader(CatHttpConstants.CAT_HTTP_HEADER_CHILD_MESSAGE_ID));
catContext.addProperty(Cat.Context.PARENT, request.getHeader(CatHttpConstants.CAT_HTTP_HEADER_PARENT_MESSAGE_ID));
catContext.addProperty(Cat.Context.ROOT, request.getHeader(CatHttpConstants.CAT_HTTP_HEADER_ROOT_MESSAGE_ID));
Cat.logRemoteCallServer(catContext);
Transaction t = Cat.newTransaction(com.dianping.cat.CatConstants.TYPE_URL, url);
try {
//日志记录
Cat.logEvent("service.method",request.getMethod(), Message.SUCCESS, request.getRequestURL().toString());
Cat.logEvent("Service.client", request.getRemoteHost());
filterChain.doFilter(servletRequest, servletResponse);
//设置事务状态为 SUCCESS
t.setStatus(Transaction.SUCCESS);
}catch (Exception e){
//设置事务状态,打印日志
t.setStatus(e);
Cat.logError(e);
throw e;
}finally {
//事务完成
t.complete();
}
}
}
CatFilterConfigure
:过滤器配置类
@Configuration
public
class
CatFilterConfigure {
@Bean
public FilterRegistrationBean catFilter(){
FilterRegistrationBean
registration
=
new
FilterRegistrationBean();
CatServletFilter
filter
=
new
CatServletFilter();
registration.setFilter(filter);
registration.addUrlPatterns("/*");
registration.setName("cat-filter");
registration.setOrder(1);
return registration;
}
}
CatRestInterceptor
:Cat拦截器 记录 TID、PID、SID
@Component
public class CatRestInterceptor implements ClientHttpRequestInterceptor {
@Override
public ClientHttpResponse intercept(HttpRequest httpRequest, byte[] bytes, ClientHttpRequestExecution clientHttpRequestExecution) throws IOException {
Transaction t = Cat.newTransaction(CatConstants.TYPE_CALL, httpRequest.getURI().toString());
try {
HttpHeaders headers = httpRequest.getHeaders();
// 保存和传递CAT调用链上下文
Cat.Context ctx = new CatContext();
Cat.logRemoteCallClient(ctx);
headers.add(CatHttpConstants.CAT_HTTP_HEADER_ROOT_MESSAGE_ID, ctx.getProperty(Cat.Context.ROOT));
headers.add(CatHttpConstants.CAT_HTTP_HEADER_PARENT_MESSAGE_ID, ctx.getProperty(Cat.Context.PARENT));
headers.add(CatHttpConstants.CAT_HTTP_HEADER_CHILD_MESSAGE_ID, ctx.getProperty(Cat.Context.CHILD));
// 继续执行请求
ClientHttpResponse response = clientHttpRequestExecution.execute(httpRequest, bytes);
t.setStatus(Transaction.SUCCESS);
return response;
} catch (Exception e) {
Cat.getProducer().logError(e);
t.setStatus(e);
throw e;
} finally {
t.complete();
}
}
}
CatHttpConstants
:cat常量
public
class
CatHttpConstants {
public
static
final
String CAT_HTTP_HEADER_CHILD_MESSAGE_ID = "X-CAT-CHILD-ID";
public
static
final
String CAT_HTTP_HEADER_PARENT_MESSAGE_ID= "X-CAT-PARENT-ID";
public
static
final
String CAT_HTTP_HEADER_ROOT_MESSAGE_ID = "X-CAT-ROOT-ID";
}
app.name:
cat-gateway
父级为
cat-demo
,命名为cat-account
启动类
@SpringBootApplication
@EnableDiscoveryClient
@RestController
@ComponentScan("com.xiaobear")
public class CatAccountApplication {
public static void main(String[] args) {
SpringApplication.run(CatAccountApplication.class, args);
}
/**
* 提供账户服务接口
* @return
* @throws Exception
*/
@RequestMapping("/account")
public String account() throws Exception {
Thread.sleep(150);
return "account success";
}
}
其他均跟网关服务类似,不同点:
CatRestInterceptor
:Cat拦截器 记录 TID、PID、SID父级为
cat-demo
,命名为cat-order
启动类
@SpringBootApplication
@EnableDiscoveryClient
@RestController
@ComponentScan("com.xiaobear")
@Slf4j
public class CatOrderApplication {
public static void main(String[] args) {
SpringApplication.run(CatOrderApplication.class, args);
}
@Autowired
RestTemplate restTemplate;
/**
* Account账户服务
*/
@Value("${service3.address:localhost:8083}")
String serviceAddress3;
/**
* stock库存服务
*/
@Value("${service4.address:localhost:8084}")
String serviceAddress4;
/**
* 异常测试端口
*/
private static final int MOCK_PORT = 8765;
/**
* 提供下单接口
* @return
* @throws InterruptedException
*/
@RequestMapping("/order")
public String service2MethodInController() throws InterruptedException {
Thread.sleep(200);
// 调用账户服务以及库存服务
String service3 = restTemplate.getForObject("http://" + serviceAddress3 + "/account", String.class);
String service4 = restTemplate.getForObject("http://" + serviceAddress4 + "/stock", String.class);
//打印返回结果
return String.format("Calling order service[order success] ==> Calling Account Service [%s] ==> Calling Customer Service [%s]", service3, service4);
}
/**
* 模拟异常超时接口
* @return
* @throws InterruptedException
*/
@RequestMapping("/readtimeout")
public String connectionTimeout() throws InterruptedException {
Transaction t = Cat.newTransaction(CatConstants.TYPE_CALL, "connectionTimeout");
//睡眠500ms
Thread.sleep(500);
try {
log.info("Calling a missing service");
//远程调用异常端口
restTemplate.getForObject("http://localhost:" + MOCK_PORT + "/readtimeout", String.class);
return "Should blow up";
} catch(Exception e) {
t.setStatus(e);
Cat.getProducer().logError(e);
throw e;
} finally {
t.complete();
}
}
@Bean
RestTemplate restTemplate() {
SimpleClientHttpRequestFactory clientHttpRequestFactory = new SimpleClientHttpRequestFactory();
clientHttpRequestFactory.setConnectTimeout(2000);
clientHttpRequestFactory.setReadTimeout(3000);
RestTemplate restTemplate = new RestTemplate(clientHttpRequestFactory);
// 保存和传递调用链上下文
restTemplate.setInterceptors(Collections.singletonList(new CatRestInterceptor()));
restTemplate.setErrorHandler(new DefaultResponseErrorHandler() {
@Override public boolean hasError(ClientHttpResponse response)
throws IOException {
try {
return super.hasError(response);
} catch (Exception e) {
return true;
}
}
@Override public void handleError(ClientHttpResponse response)
throws IOException {
try {
super.handleError(response);
} catch (Exception e) {
log.error("Exception [" + e.getMessage() + "] occurred while trying to send the request", e);
throw e;
}
}
});
return restTemplate;
}
}
其他均跟网关服务类似,不同点:
父级为
cat-demo
,命名为cat-stock
启动类
@SpringBootApplication
@EnableDiscoveryClient
@ComponentScan("com.xiaobear")
@RestController
public class CatStockApplication {
public static void main(String[] args) {
SpringApplication.run(CatStockApplication.class, args);
}
/**
* 提供库存接口
* @return
* @throws Exception
*/
@RequestMapping("/stock")
public String stock() throws Exception {
Thread.sleep(200);
return "stock success";
}
}
其他均跟网关服务类似,不同点:
CatRestInterceptor
:Cat拦截器 记录 TID、PID、SID gateway service ==> Calling order service[order success] ==> Calling Account Service [account success] ==> Calling Customer Service [stock success]
CAT的LOGVIEW按层级完整的记录了四个服务的请求信息, 1至4分别对应Gateway、Order、Account和Stock服务。
LOGVIEW主要包含请求时间, 服务地址, 请求客户端等主要信息, 也支持图形方式呈现:
CAT 还有很多指标统计与报表展示, 能有效帮助我们监控管理整体微服务调用链路状态,更多大家可自行去研究哦。
如果觉得内容不错的话,希望大家可以帮忙点赞转发一波,这是对我最大的鼓励,感谢🙏🏻
END
扫码关注腾讯云开发者
领取腾讯云代金券
Copyright © 2013 - 2025 Tencent Cloud. All Rights Reserved. 腾讯云 版权所有
深圳市腾讯计算机系统有限公司 ICP备案/许可证号:粤B2-20090059 深公网安备号 44030502008569
腾讯云计算(北京)有限责任公司 京ICP证150476号 | 京ICP备11018762号 | 京公网安备号11010802020287
Copyright © 2013 - 2025 Tencent Cloud.
All Rights Reserved. 腾讯云 版权所有