[go: up one dir, main page]
Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

MQ里面调用redis，MQ的日志打印traceId正常，紧跟着redis set命令打印traceId丢失 #441

Open
KYV365 opened this issue Jun 26, 2021 · 6 comments
Open

MQ里面调用redis，MQ的日志打印traceId正常，紧跟着redis set命令打印traceId丢失 #441

KYV365 opened this issue Jun 26, 2021 · 6 comments
Labels
waiting for feedback waiting for feedback

Comments

@KYV365
Copy link
KYV365 commented Jun 26, 2021

Your question

MQ里面调用redis，MQ的日志打印traceId正常，紧跟着redis set命令打印traceId丢失

redisTemplate.opsForValue().set("test mq redis", "test mq redis");
log.info("test mq redis end 1");
log日志的traceId丢失，断点进入redis的插件，里面traceId是正常封装
describe your question clearly

Your scenes

image
@StreamListener(value = MQ_TENANT_DB_CHANNEL, condition = SYNC_TENANT_DB_CONDITION)
public void syncTenantBasicData(@payload TenantDbMessageDTO tenantDbMessageDTO, @Header(MQ_HEADER_APPCODE) String appCode) throws InterruptedException {

    // 省略xxx
    log.info("同步租户ID[{}]的DB数据，应用名[{}]与应用代号[{}]匹对，消费消息", tenantDbMessageDTO.getTenantId(), applicationName, appCode);

    log.info("test mq redis start ");
    redisTemplate.opsForValue().set("test mq redis", "test mq redis");
    log.info("test mq redis end 1");
    redisTemplate.opsForValue().set("test mq redis1", "test mq redis1");
    log.info("test mq redis end 2");
    redisTemplate.opsForValue().set("test mq redis2", "test mq redis2");
    log.info("test mq redis end 3");
    redisTemplate.opsForValue().set("test mq redis3", "test mq redis3");
    log.info("test mq redis end 4");
     // 省略xxx
    //自定义回调
    log.info("同步租户ID[{}]应用[{}]的DB数据，执行自定义回调实例条数：{}", tenantDbMessageDTO.getTenantId(), 
}

describe your use scenes (why need this feature)

Your advice

describe the advice or solution you'd like

Environment

  • SOFATracer version:
  • JVM version (e.g. java -version):
  • OS version (e.g. uname -a):
  • Maven version:
  • IDE version:
@glmapper
Copy link
Contributor
glmapper commented Jul 5, 2021

@raojun0504 当前使用的 SOFAtracer 版本是多少？可以看下 release node ,以确认当前版本是否支持 redis 客户端埋点

@glmapper glmapper added the waiting for feedback waiting for feedback label Jul 5, 2021
@KYV365
Copy link
Author
KYV365 commented Jul 8, 2021

@raojun0504 当前使用的 SOFAtracer 版本是多少？可以看下 release node ,以确认当前版本是否支持 redis 客户端埋点

image
查看了是支持的

@glmapper
Copy link
Contributor

@raojun0504 可以debug 看下 MDCSpanExtension 是否有被执行到

@keith-java
Copy link

这个问题你找到解决办法没有，我这边也遇到了经过reids操作以后traceId丢失问题

@keith-java
Copy link
keith-java commented Aug 11, 2023 •
edited
Loading

从我当前debug的结果来看，在同一个线程情况下只要操作了redis就会导致MDC的数据丢失，经过RedisActionWrapperHelper会触发获取数据后清理动作，导致MDC丢失
1：执行清理的代码入口
企业微信截图_16917368601430
2：清理MDC代码入口
企业微信截图_16917369123974
3：清理以后在设置parent的导致（真实原因应该是redis生成context的时候把父类丢失了）
image
4：最后找到问题根因在处理依赖关系导致threadlocal中数据异常，修复方法：
com.sofa.alipay.tracer.plugins.spring.redis.common.RedisActionWrapperHelper#buildSpan(java.lang.String)

image

@xyohn
Copy link
xyohn commented Nov 29, 2023

@raojun0504 可以debug 看下 MDCSpanExtension 是否有被执行到

我们这边也遇到了这个问题，排查了一下发现是com.sofa.alipay.tracer.plugins.spring.redis.common.RedisActionWrapperHelper里逻辑的问题
其他的plugin关于com.alipay.common.tracer.core.span.SofaTracerSpan对象的产生和销毁都是基于com.alipay.common.tracer.core.tracer.AbstractTracer#clientSend和com.alipay.common.tracer.core.tracer.AbstractTracer#clientReceive去实现的，在clientSend里面的逻辑包括了对父span的处理，即

            // Need to actively cache your own serverSpan, because: asChildOf is concerned about spanContext
            clientSpan.setParentSofaTracerSpan(serverSpan);

image

但是在com.sofa.alipay.tracer.plugins.spring.redis.common.RedisActionWrapperHelper中，关于com.alipay.common.tracer.core.span.SofaTracerSpan对象的产生和销毁，是在com.sofa.alipay.tracer.plugins.spring.redis.common.RedisActionWrapperHelper#builder中直接调用了com.alipay.common.tracer.core.SofaTracer#buildSpan，这个逻辑是非常奇怪的，在整个项目的plugin中，只有com.alipay.sofa.tracer.plugin.flexible.FlexibleTracer和com.sofa.alipay.tracer.plugins.spring.redis.common.RedisActionWrapperHelper是这么做的。
image
然而，com.alipay.sofa.tracer.plugin.flexible.FlexibleTracer机智的把父span关联的逻辑补上了
image
但com.sofa.alipay.tracer.plugins.spring.redis.common.RedisActionWrapperHelper中没有，导致Redis的plugin存在问题。
按我的理解，所有的plugin应该只是在兼容各组件上存在差异，在span相关的逻辑应该是保持一致的，基于此才会有com.alipay.common.tracer.core.tracer.AbstractTracer的抽象类存在。
不清楚为什么Redis的另辟蹊径使用自有的方式来实现，关键逻辑上还存在问题..😂

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
waiting for feedback waiting for feedback
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@xyohn @keith-java @glmapper @KYV365