Multithread processing with RedisQueueInboundGateway
up vote
1
down vote
favorite
I'm using Spring Integration with Redis. The producer uses RedisQueueOutboundGateway and on the other side the receiver have a flow defined with RedisQueueInboundGateway.
Reading from the documentation I found the following sentence
The task-executor has to be configured with more than one thread for processing
My need is to have concurrent executions, in order to speed up the elaboration of requests, but I can see there is always one thread even if I configured a custom ThreadPoolTaskExecutor like the following
public Executor getAsyncExecutor() {
ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
executor.setCorePoolSize(5);
executor.setMaxPoolSize(40);
executor.setQueueCapacity(40);
executor.setThreadNamePrefix("QueueAsyncExecutor-");
executor.initialize();
return executor;
}
and the use of this threadpool is
final RedisQueueInboundGateway rqig = new RedisQueueInboundGateway(finalDestination, jedisConnectionFactory);
rqig.setTaskExecutor(getAsyncExecutor());
The final result is a sequential processing of the requests, all done with the same thread as i can see from the log. Is it possible to enable the multithread processing in that situation? How?
spring multithreading redis spring-integration
asked Nov 20 at 14:52
Federico Paparoni
18914
add a comment |
up vote
1
down vote
favorite
I'm using Spring Integration with Redis. The producer uses RedisQueueOutboundGateway and on the other side the receiver have a flow defined with RedisQueueInboundGateway.
Reading from the documentation I found the following sentence
The task-executor has to be configured with more than one thread for processing
My need is to have concurrent executions, in order to speed up the elaboration of requests, but I can see there is always one thread even if I configured a custom ThreadPoolTaskExecutor like the following
public Executor getAsyncExecutor() {
ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
executor.setCorePoolSize(5);
executor.setMaxPoolSize(40);
executor.setQueueCapacity(40);
executor.setThreadNamePrefix("QueueAsyncExecutor-");
executor.initialize();
return executor;
}
and the use of this threadpool is
final RedisQueueInboundGateway rqig = new RedisQueueInboundGateway(finalDestination, jedisConnectionFactory);
rqig.setTaskExecutor(getAsyncExecutor());
The final result is a sequential processing of the requests, all done with the same thread as i can see from the log. Is it possible to enable the multithread processing in that situation? How?
spring multithreading redis spring-integration
asked Nov 20 at 14:52
Federico Paparoni
18914
add a comment |
up vote
1
down vote
favorite
up vote
1
down vote
favorite
I'm using Spring Integration with Redis. The producer uses RedisQueueOutboundGateway and on the other side the receiver have a flow defined with RedisQueueInboundGateway.
Reading from the documentation I found the following sentence
The task-executor has to be configured with more than one thread for processing
My need is to have concurrent executions, in order to speed up the elaboration of requests, but I can see there is always one thread even if I configured a custom ThreadPoolTaskExecutor like the following
public Executor getAsyncExecutor() {
ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
executor.setCorePoolSize(5);
executor.setMaxPoolSize(40);
executor.setQueueCapacity(40);
executor.setThreadNamePrefix("QueueAsyncExecutor-");
executor.initialize();
return executor;
}
and the use of this threadpool is
final RedisQueueInboundGateway rqig = new RedisQueueInboundGateway(finalDestination, jedisConnectionFactory);
rqig.setTaskExecutor(getAsyncExecutor());
The final result is a sequential processing of the requests, all done with the same thread as i can see from the log. Is it possible to enable the multithread processing in that situation? How?
spring multithreading redis spring-integration
asked Nov 20 at 14:52
Federico Paparoni
18914
I'm using Spring Integration with Redis. The producer uses RedisQueueOutboundGateway and on the other side the receiver have a flow defined with RedisQueueInboundGateway.
Reading from the documentation I found the following sentence
The task-executor has to be configured with more than one thread for processing
My need is to have concurrent executions, in order to speed up the elaboration of requests, but I can see there is always one thread even if I configured a custom ThreadPoolTaskExecutor like the following
public Executor getAsyncExecutor() {
ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
executor.setCorePoolSize(5);
executor.setMaxPoolSize(40);
executor.setQueueCapacity(40);
executor.setThreadNamePrefix("QueueAsyncExecutor-");
executor.initialize();
return executor;
}
and the use of this threadpool is
final RedisQueueInboundGateway rqig = new RedisQueueInboundGateway(finalDestination, jedisConnectionFactory);
rqig.setTaskExecutor(getAsyncExecutor());
The final result is a sequential processing of the requests, all done with the same thread as i can see from the log. Is it possible to enable the multithread processing in that situation? How?
spring multithreading redis spring-integration
spring multithreading redis spring-integration
asked Nov 20 at 14:52
Federico Paparoni
18914
asked Nov 20 at 14:52
Federico Paparoni
18914
asked Nov 20 at 14:52
Federico Paparoni
18914
asked Nov 20 at 14:52
Federico Paparoni
18914
asked Nov 20 at 14:52
Federico Paparoni
18914
18914
add a comment |
add a comment |
1 Answer
1
active
oldest
votes
up vote
1
down vote
accepted
That's correct. The RedisQueueInboundGateway is a single-threaded for now. There is only one ListenerTask:
private void restart() {
this.taskExecutor.execute(new ListenerTask());
}
Sounds like we need to introduce concurrency option into that RedisQueueInboundGateway! Feel free to raise a JIRA on the matter and contribution is welcome!
You may achieve an artificial concurrency with several RedisQueueInboundGateway instances for the same Redis queue. This way each of them will start its own ListenerTask.
answered Nov 20 at 15:12
Artem Bilan
63.5k84668
Thank you for the quick response. Is it correct to simulate the concurrency the use of RedisInboundChannelAdapter (that can be multi-thread) and finally with a custom flow adding a message to redis with ".reply", so the caller with RedisQueueOutboundGateway can read it ?
– Federico Paparoni
Nov 20 at 15:27
1
Yes, you can play with that as well. Only the problem thatRedisQueueInboundGatewayhas some hard logic around UUIDs for requests to pop.
– Artem Bilan
Nov 20 at 15:37
Taking advantage of your kindness, can you tell me why I found Redis Queue Inbound Channel Adapter in the documentation and it's available as xml entity (int-redis:queue-inbound-channel-adapter) but I didn't find a Java DSL equivalent
– Federico Paparoni
Nov 20 at 16:51
Because it hasn't been implemented yet: jira.spring.io/browse/INT-4165
– Artem Bilan
Nov 20 at 16:54
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53395644%2fmultithread-processing-with-redisqueueinboundgateway%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
up vote
1
down vote
accepted
That's correct. The RedisQueueInboundGateway is a single-threaded for now. There is only one ListenerTask:
private void restart() {
this.taskExecutor.execute(new ListenerTask());
}
Sounds like we need to introduce concurrency option into that RedisQueueInboundGateway! Feel free to raise a JIRA on the matter and contribution is welcome!
You may achieve an artificial concurrency with several RedisQueueInboundGateway instances for the same Redis queue. This way each of them will start its own ListenerTask.
answered Nov 20 at 15:12
Artem Bilan
63.5k84668
Thank you for the quick response. Is it correct to simulate the concurrency the use of RedisInboundChannelAdapter (that can be multi-thread) and finally with a custom flow adding a message to redis with ".reply", so the caller with RedisQueueOutboundGateway can read it ?
– Federico Paparoni
Nov 20 at 15:27
1
Yes, you can play with that as well. Only the problem thatRedisQueueInboundGatewayhas some hard logic around UUIDs for requests to pop.
– Artem Bilan
Nov 20 at 15:37
Taking advantage of your kindness, can you tell me why I found Redis Queue Inbound Channel Adapter in the documentation and it's available as xml entity (int-redis:queue-inbound-channel-adapter) but I didn't find a Java DSL equivalent
– Federico Paparoni
Nov 20 at 16:51
Because it hasn't been implemented yet: jira.spring.io/browse/INT-4165
– Artem Bilan
Nov 20 at 16:54
add a comment |
up vote
1
down vote
accepted
That's correct. The RedisQueueInboundGateway is a single-threaded for now. There is only one ListenerTask:
private void restart() {
this.taskExecutor.execute(new ListenerTask());
}
Sounds like we need to introduce concurrency option into that RedisQueueInboundGateway! Feel free to raise a JIRA on the matter and contribution is welcome!
You may achieve an artificial concurrency with several RedisQueueInboundGateway instances for the same Redis queue. This way each of them will start its own ListenerTask.
answered Nov 20 at 15:12
Artem Bilan
63.5k84668
Thank you for the quick response. Is it correct to simulate the concurrency the use of RedisInboundChannelAdapter (that can be multi-thread) and finally with a custom flow adding a message to redis with ".reply", so the caller with RedisQueueOutboundGateway can read it ?
– Federico Paparoni
Nov 20 at 15:27
1
Yes, you can play with that as well. Only the problem thatRedisQueueInboundGatewayhas some hard logic around UUIDs for requests to pop.
– Artem Bilan
Nov 20 at 15:37
Taking advantage of your kindness, can you tell me why I found Redis Queue Inbound Channel Adapter in the documentation and it's available as xml entity (int-redis:queue-inbound-channel-adapter) but I didn't find a Java DSL equivalent
– Federico Paparoni
Nov 20 at 16:51
Because it hasn't been implemented yet: jira.spring.io/browse/INT-4165
– Artem Bilan
Nov 20 at 16:54
add a comment |
up vote
1
down vote
accepted
up vote
1
down vote
accepted
That's correct. The RedisQueueInboundGateway is a single-threaded for now. There is only one ListenerTask:
private void restart() {
this.taskExecutor.execute(new ListenerTask());
}
Sounds like we need to introduce concurrency option into that RedisQueueInboundGateway! Feel free to raise a JIRA on the matter and contribution is welcome!
You may achieve an artificial concurrency with several RedisQueueInboundGateway instances for the same Redis queue. This way each of them will start its own ListenerTask.
answered Nov 20 at 15:12
Artem Bilan
63.5k84668
That's correct. The RedisQueueInboundGateway is a single-threaded for now. There is only one ListenerTask:
private void restart() {
this.taskExecutor.execute(new ListenerTask());
}
Sounds like we need to introduce concurrency option into that RedisQueueInboundGateway! Feel free to raise a JIRA on the matter and contribution is welcome!
You may achieve an artificial concurrency with several RedisQueueInboundGateway instances for the same Redis queue. This way each of them will start its own ListenerTask.
answered Nov 20 at 15:12
Artem Bilan
63.5k84668
answered Nov 20 at 15:12
Artem Bilan
63.5k84668
answered Nov 20 at 15:12
Artem Bilan
63.5k84668
answered Nov 20 at 15:12
Artem Bilan
63.5k84668
63.5k84668
Thank you for the quick response. Is it correct to simulate the concurrency the use of RedisInboundChannelAdapter (that can be multi-thread) and finally with a custom flow adding a message to redis with ".reply", so the caller with RedisQueueOutboundGateway can read it ?
– Federico Paparoni
Nov 20 at 15:27
1
Yes, you can play with that as well. Only the problem thatRedisQueueInboundGatewayhas some hard logic around UUIDs for requests to pop.
– Artem Bilan
Nov 20 at 15:37
Taking advantage of your kindness, can you tell me why I found Redis Queue Inbound Channel Adapter in the documentation and it's available as xml entity (int-redis:queue-inbound-channel-adapter) but I didn't find a Java DSL equivalent
– Federico Paparoni
Nov 20 at 16:51
Because it hasn't been implemented yet: jira.spring.io/browse/INT-4165
– Artem Bilan
Nov 20 at 16:54
add a comment |
Thank you for the quick response. Is it correct to simulate the concurrency the use of RedisInboundChannelAdapter (that can be multi-thread) and finally with a custom flow adding a message to redis with ".reply", so the caller with RedisQueueOutboundGateway can read it ?
– Federico Paparoni
Nov 20 at 15:27
1
Yes, you can play with that as well. Only the problem thatRedisQueueInboundGatewayhas some hard logic around UUIDs for requests to pop.
– Artem Bilan
Nov 20 at 15:37
Taking advantage of your kindness, can you tell me why I found Redis Queue Inbound Channel Adapter in the documentation and it's available as xml entity (int-redis:queue-inbound-channel-adapter) but I didn't find a Java DSL equivalent
– Federico Paparoni
Nov 20 at 16:51
Because it hasn't been implemented yet: jira.spring.io/browse/INT-4165
– Artem Bilan
Nov 20 at 16:54
Thank you for the quick response. Is it correct to simulate the concurrency the use of RedisInboundChannelAdapter (that can be multi-thread) and finally with a custom flow adding a message to redis with ".reply", so the caller with RedisQueueOutboundGateway can read it ?
– Federico Paparoni
Nov 20 at 15:27
Thank you for the quick response. Is it correct to simulate the concurrency the use of RedisInboundChannelAdapter (that can be multi-thread) and finally with a custom flow adding a message to redis with ".reply", so the caller with RedisQueueOutboundGateway can read it ?
– Federico Paparoni
Nov 20 at 15:27
1
1
Yes, you can play with that as well. Only the problem that
RedisQueueInboundGateway has some hard logic around UUIDs for requests to pop.– Artem Bilan
Nov 20 at 15:37
Yes, you can play with that as well. Only the problem that
RedisQueueInboundGateway has some hard logic around UUIDs for requests to pop.– Artem Bilan
Nov 20 at 15:37
Taking advantage of your kindness, can you tell me why I found Redis Queue Inbound Channel Adapter in the documentation and it's available as xml entity (int-redis:queue-inbound-channel-adapter) but I didn't find a Java DSL equivalent
– Federico Paparoni
Nov 20 at 16:51
Taking advantage of your kindness, can you tell me why I found Redis Queue Inbound Channel Adapter in the documentation and it's available as xml entity (int-redis:queue-inbound-channel-adapter) but I didn't find a Java DSL equivalent
– Federico Paparoni
Nov 20 at 16:51
Because it hasn't been implemented yet: jira.spring.io/browse/INT-4165
– Artem Bilan
Nov 20 at 16:54
Because it hasn't been implemented yet: jira.spring.io/browse/INT-4165
– Artem Bilan
Nov 20 at 16:54
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Some of your past answers have not been well-received, and you're in danger of being blocked from answering.
Please pay close attention to the following guidance:
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53395644%2fmultithread-processing-with-redisqueueinboundgateway%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
