Fastest queries in Postgres with Node.js client and connection pooling - stored functions or prepared...
We have a Postgres database on AWS, with a table that will contain millions (but not tens of millions) of rows. Primary key is a single column. Data access is going to be basically uniformly distributed among the data rows, and it's going to be simple SELECT statements by primary key, so basically we get either one row or none as the result data set.
Our client is written in Node.js using the node-postgres module. We are using connection pooling. We would like to do anything we reasonably can to execute our SELECTs as fast as possible; would prepared statements or stored functions be a good idea in this scenario? What would be, on paper, faster?
Edit: we are currently handling connection pooling via the pg-pool module that comes with node-psotgres itself, and pick a new connection from the pool with every request. I am not sure whether the work done by the server when preparing a statement, or a stored function, is lost when the connection is handed back to the pool.
sql node.js postgresql query-optimization
add a comment |
We have a Postgres database on AWS, with a table that will contain millions (but not tens of millions) of rows. Primary key is a single column. Data access is going to be basically uniformly distributed among the data rows, and it's going to be simple SELECT statements by primary key, so basically we get either one row or none as the result data set.
Our client is written in Node.js using the node-postgres module. We are using connection pooling. We would like to do anything we reasonably can to execute our SELECTs as fast as possible; would prepared statements or stored functions be a good idea in this scenario? What would be, on paper, faster?
Edit: we are currently handling connection pooling via the pg-pool module that comes with node-psotgres itself, and pick a new connection from the pool with every request. I am not sure whether the work done by the server when preparing a statement, or a stored function, is lost when the connection is handed back to the pool.
sql node.js postgresql query-optimization
add a comment |
We have a Postgres database on AWS, with a table that will contain millions (but not tens of millions) of rows. Primary key is a single column. Data access is going to be basically uniformly distributed among the data rows, and it's going to be simple SELECT statements by primary key, so basically we get either one row or none as the result data set.
Our client is written in Node.js using the node-postgres module. We are using connection pooling. We would like to do anything we reasonably can to execute our SELECTs as fast as possible; would prepared statements or stored functions be a good idea in this scenario? What would be, on paper, faster?
Edit: we are currently handling connection pooling via the pg-pool module that comes with node-psotgres itself, and pick a new connection from the pool with every request. I am not sure whether the work done by the server when preparing a statement, or a stored function, is lost when the connection is handed back to the pool.
sql node.js postgresql query-optimization
We have a Postgres database on AWS, with a table that will contain millions (but not tens of millions) of rows. Primary key is a single column. Data access is going to be basically uniformly distributed among the data rows, and it's going to be simple SELECT statements by primary key, so basically we get either one row or none as the result data set.
Our client is written in Node.js using the node-postgres module. We are using connection pooling. We would like to do anything we reasonably can to execute our SELECTs as fast as possible; would prepared statements or stored functions be a good idea in this scenario? What would be, on paper, faster?
Edit: we are currently handling connection pooling via the pg-pool module that comes with node-psotgres itself, and pick a new connection from the pool with every request. I am not sure whether the work done by the server when preparing a statement, or a stored function, is lost when the connection is handed back to the pool.
sql node.js postgresql query-optimization
sql node.js postgresql query-optimization
edited Nov 22 '18 at 10:21
Btz
asked Nov 21 '18 at 14:44
BtzBtz
135418
135418
add a comment |
add a comment |
1 Answer
1
active
oldest
votes
You might optimize for PostgreSQL query performance on Node.js a couple ways with the node-postgres module, but with the use cases you describe, I feel like neither of them would offer significant performance benefits. The general tl;dr: is to benchmark, benchmark, benchmark.
Prepared Statements
Using prepared statements and a connection pool are not mutually exclusive, but the query plans for prepared statements are only cached on a per-connection basis. As long as those connections remain active (as long as the Pool itself is not closed), the cached query plans remain intact and associated with the connection that executed them. In the worst-case scenario, multiple connections perform the planning stage for a given prepared query once each.
Prepared statements may confer some performance benefits, since node-postgres does skip the query planning phase for them after first execution (source, JavaScript, source, native bindings).
Native Bindings
And speaking of the native bindings, the author of the node-postgres module also maintains a Node.js-compatible set of bindings to the native C libpq library in the package node-pg-native. If your deployment environment supports a compatible compiler, they might be worth investigating.
The documentation warns that there may be edge case incompatibilities between the JavaScript client and the native bindings, so be sure to take those into consideration before you make such a transition to ensure nothing else in your application breaks.
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53414537%2ffastest-queries-in-postgres-with-node-js-client-and-connection-pooling-stored%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
You might optimize for PostgreSQL query performance on Node.js a couple ways with the node-postgres module, but with the use cases you describe, I feel like neither of them would offer significant performance benefits. The general tl;dr: is to benchmark, benchmark, benchmark.
Prepared Statements
Using prepared statements and a connection pool are not mutually exclusive, but the query plans for prepared statements are only cached on a per-connection basis. As long as those connections remain active (as long as the Pool itself is not closed), the cached query plans remain intact and associated with the connection that executed them. In the worst-case scenario, multiple connections perform the planning stage for a given prepared query once each.
Prepared statements may confer some performance benefits, since node-postgres does skip the query planning phase for them after first execution (source, JavaScript, source, native bindings).
Native Bindings
And speaking of the native bindings, the author of the node-postgres module also maintains a Node.js-compatible set of bindings to the native C libpq library in the package node-pg-native. If your deployment environment supports a compatible compiler, they might be worth investigating.
The documentation warns that there may be edge case incompatibilities between the JavaScript client and the native bindings, so be sure to take those into consideration before you make such a transition to ensure nothing else in your application breaks.
add a comment |
You might optimize for PostgreSQL query performance on Node.js a couple ways with the node-postgres module, but with the use cases you describe, I feel like neither of them would offer significant performance benefits. The general tl;dr: is to benchmark, benchmark, benchmark.
Prepared Statements
Using prepared statements and a connection pool are not mutually exclusive, but the query plans for prepared statements are only cached on a per-connection basis. As long as those connections remain active (as long as the Pool itself is not closed), the cached query plans remain intact and associated with the connection that executed them. In the worst-case scenario, multiple connections perform the planning stage for a given prepared query once each.
Prepared statements may confer some performance benefits, since node-postgres does skip the query planning phase for them after first execution (source, JavaScript, source, native bindings).
Native Bindings
And speaking of the native bindings, the author of the node-postgres module also maintains a Node.js-compatible set of bindings to the native C libpq library in the package node-pg-native. If your deployment environment supports a compatible compiler, they might be worth investigating.
The documentation warns that there may be edge case incompatibilities between the JavaScript client and the native bindings, so be sure to take those into consideration before you make such a transition to ensure nothing else in your application breaks.
add a comment |
You might optimize for PostgreSQL query performance on Node.js a couple ways with the node-postgres module, but with the use cases you describe, I feel like neither of them would offer significant performance benefits. The general tl;dr: is to benchmark, benchmark, benchmark.
Prepared Statements
Using prepared statements and a connection pool are not mutually exclusive, but the query plans for prepared statements are only cached on a per-connection basis. As long as those connections remain active (as long as the Pool itself is not closed), the cached query plans remain intact and associated with the connection that executed them. In the worst-case scenario, multiple connections perform the planning stage for a given prepared query once each.
Prepared statements may confer some performance benefits, since node-postgres does skip the query planning phase for them after first execution (source, JavaScript, source, native bindings).
Native Bindings
And speaking of the native bindings, the author of the node-postgres module also maintains a Node.js-compatible set of bindings to the native C libpq library in the package node-pg-native. If your deployment environment supports a compatible compiler, they might be worth investigating.
The documentation warns that there may be edge case incompatibilities between the JavaScript client and the native bindings, so be sure to take those into consideration before you make such a transition to ensure nothing else in your application breaks.
You might optimize for PostgreSQL query performance on Node.js a couple ways with the node-postgres module, but with the use cases you describe, I feel like neither of them would offer significant performance benefits. The general tl;dr: is to benchmark, benchmark, benchmark.
Prepared Statements
Using prepared statements and a connection pool are not mutually exclusive, but the query plans for prepared statements are only cached on a per-connection basis. As long as those connections remain active (as long as the Pool itself is not closed), the cached query plans remain intact and associated with the connection that executed them. In the worst-case scenario, multiple connections perform the planning stage for a given prepared query once each.
Prepared statements may confer some performance benefits, since node-postgres does skip the query planning phase for them after first execution (source, JavaScript, source, native bindings).
Native Bindings
And speaking of the native bindings, the author of the node-postgres module also maintains a Node.js-compatible set of bindings to the native C libpq library in the package node-pg-native. If your deployment environment supports a compatible compiler, they might be worth investigating.
The documentation warns that there may be edge case incompatibilities between the JavaScript client and the native bindings, so be sure to take those into consideration before you make such a transition to ensure nothing else in your application breaks.
edited Nov 29 '18 at 18:13
answered Nov 28 '18 at 23:30
ConnorConnor
72221021
72221021
add a comment |
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Some of your past answers have not been well-received, and you're in danger of being blocked from answering.
Please pay close attention to the following guidance:
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53414537%2ffastest-queries-in-postgres-with-node-js-client-and-connection-pooling-stored%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown