Python库调用c++库的流程
Python库 调用C++库的流程
python 库创建consumer的过程:
self.consumer = PushConsumer(self.consumer_group_id)
self.consumer.subscribe(self.topic, self.callback_func)
self.consumer.start()
PushConsumer有接收三个参数,init 的原型为:
def __init__(self, group_id, orderly=False, message_model=MessageModel.CLUSTERING):
self._handle = dll.CreatePushConsumer(_to_bytes(group_id))
if self._handle is None:
raise NullPointerException('Returned null pointer when create PushConsumer')
self._orderly = orderly
self.set_message_model(message_model)
self._callback_refs = []
其中:
- group_id 表示消费者组
- orderly 表示是否为顺序消费
- message_model 默认为集群消费模式,还有一种消费模式是广播消费模式
dll.CreatePushConsumer的源码如下:
所属文件: src/extern/CPushConsumer.cpp
CPushConsumer* CreatePushConsumer(const char* groupId) {
if (groupId == NULL) {
return NULL;
}
DefaultMQPushConsumer* defaultMQPushConsumer = new DefaultMQPushConsumer(groupId);
defaultMQPushConsumer->setConsumeFromWhere(CONSUME_FROM_LAST_OFFSET);
strncpy(VERSION_FOR_PUSH_CONSUMER, defaultMQPushConsumer->version().c_str(), MAX_SDK_VERSION_LENGTH - 1);
VERSION_FOR_PUSH_CONSUMER[MAX_SDK_VERSION_LENGTH - 1] = 0;
return (CPushConsumer*)defaultMQPushConsumer;
}
self.consumer.subscribe的执行路径: 在python 的subscribe函数中主要调用了两个方法:
ffi_check(dll.Subscribe(self._handle, _to_bytes(topic), _to_bytes(expression)))
self._register_callback(_on_message)
可以看到SubScribe函数实际执行了两个动作:
- 一个是dll.Subscribe,完成消费者订阅的动作,expression表示订阅的tags
- 另一个是注册消费回调函数
其中:dll.Subscribe方法的调用路径为:
int Subscribe(CPushConsumer* consumer, const char* topic, const char* expression) {
if (consumer == NULL) {
return NULL_POINTER;
}
((DefaultMQPushConsumer*)consumer)->subscribe(topic, expression);
return OK;
}
self._handler 为 defaultMQPushConsumer Subscribe 文件路径:src/extern/CPushConsumer.cpp 调用路径是:
self._handler->subscribe --> impl->subscribe -->
m_subTopics[topic] = subExpression;
其中m_subTopics 的定义为: std::map<std::string, std::string> m_subTopics;
self._register_callback的执行路径: step1: 先确定是RegisterMessageCallbackOrderly 还是RegisterMessageCallback, 这取决于之前的ordely参数 step2: 将callback 添加到self.__callback_refs step3: 执行step1中注册函数,参数为self._handler,和回调函数
int RegisterMessageCallback(CPushConsumer* consumer, MessageCallBack pCallback) {
if (consumer == NULL || pCallback == NULL) {
return NULL_POINTER;
}
MessageListenerInner* listenerInner = new MessageListenerInner(consumer, pCallback);
((DefaultMQPushConsumer*)consumer)->registerMessageListener(listenerInner);
g_ListenerMap[consumer] = listenerInner;
return OK;
}
self.consumer.start()的执行路径:
执行的src/extern/CPushConsumer.cpp的 StartPushConsumer
-> (DefaultMQPushConsumer*)consumer)->start() -> impl->start();
ConsumeMessageConcurrentlyService::ConsumeRequest 会实际实际执行消息的消费
try {
status = m_pMessageListener->consumeMessage(msgs);
} catch (...) {
status = RECONSUME_LATER;
LOG_ERROR("Consumer's code is buggy. Un-caught exception raised");
}
上述方法会调用Listener的consumeMessage 方法,该方法中会触发Python中的callback函数。
进行消息的消费:
ConsumeStatus consumeMessage(const std::vector<MQMessageExt>& msgs) {
// to do user call back
if (m_pMsgReceiveCallback == NULL) {
return RECONSUME_LATER;
}
for (size_t i = 0; i < msgs.size(); ++i) {
MQMessageExt* msg = const_cast<MQMessageExt*>(&msgs[i]);
CMessageExt* message = (CMessageExt*)(msg);
if (m_pMsgReceiveCallback(m_pconsumer, message) != E_CONSUME_SUCCESS)
return RECONSUME_LATER;
}
return CONSUME_SUCCESS;
}
private:
MessageCallBack m_pMsgReceiveCallback;
CPushConsumer* m_pconsumer;
};
从上述方法可以看出,如果一批消息中有一个消费失败了,就会终止后续消息的处理。