Counting number of similar strings in Postgresql
I have a table that contains a list of search_terms in Postgresql and the number of times they were searched:
Search Term Table
I'm trying to write a query that groups them together i.e. I want to see that electric scooter has been searched 27 times rather than 20 with 4 misspelt one way and 3 misspelt the other. I want to use the similarity function so that I can play with the limit.
I have been trying something along the lines of grouping by similarity but with no success:
SELECT
search_term,
SUM(count)
FROM
t2
GROUP BY (SELECT set_limit(0.8);
SELECT similarity(n1.search_term, n2.search_term) AS sim, n1.search_term, n2.search_term
FROM t2 n1
JOIN t2 n2 ON n1.search_term <> n2.search_term
AND n1.search_term % n2.search_term
ORDER BY sim DESC)
Any help greatly appreciated!
postgresql grouping similarity
asked Nov 22 '18 at 18:09
user10670868user10670868
111
add a comment |
I have a table that contains a list of search_terms in Postgresql and the number of times they were searched:
Search Term Table
I'm trying to write a query that groups them together i.e. I want to see that electric scooter has been searched 27 times rather than 20 with 4 misspelt one way and 3 misspelt the other. I want to use the similarity function so that I can play with the limit.
I have been trying something along the lines of grouping by similarity but with no success:
SELECT
search_term,
SUM(count)
FROM
t2
GROUP BY (SELECT set_limit(0.8);
SELECT similarity(n1.search_term, n2.search_term) AS sim, n1.search_term, n2.search_term
FROM t2 n1
JOIN t2 n2 ON n1.search_term <> n2.search_term
AND n1.search_term % n2.search_term
ORDER BY sim DESC)
Any help greatly appreciated!
postgresql grouping similarity
asked Nov 22 '18 at 18:09
user10670868user10670868
111
add a comment |
I have a table that contains a list of search_terms in Postgresql and the number of times they were searched:
Search Term Table
I'm trying to write a query that groups them together i.e. I want to see that electric scooter has been searched 27 times rather than 20 with 4 misspelt one way and 3 misspelt the other. I want to use the similarity function so that I can play with the limit.
I have been trying something along the lines of grouping by similarity but with no success:
SELECT
search_term,
SUM(count)
FROM
t2
GROUP BY (SELECT set_limit(0.8);
SELECT similarity(n1.search_term, n2.search_term) AS sim, n1.search_term, n2.search_term
FROM t2 n1
JOIN t2 n2 ON n1.search_term <> n2.search_term
AND n1.search_term % n2.search_term
ORDER BY sim DESC)
Any help greatly appreciated!
postgresql grouping similarity
asked Nov 22 '18 at 18:09
user10670868user10670868
111
I have a table that contains a list of search_terms in Postgresql and the number of times they were searched:
Search Term Table
I'm trying to write a query that groups them together i.e. I want to see that electric scooter has been searched 27 times rather than 20 with 4 misspelt one way and 3 misspelt the other. I want to use the similarity function so that I can play with the limit.
I have been trying something along the lines of grouping by similarity but with no success:
SELECT
search_term,
SUM(count)
FROM
t2
GROUP BY (SELECT set_limit(0.8);
SELECT similarity(n1.search_term, n2.search_term) AS sim, n1.search_term, n2.search_term
FROM t2 n1
JOIN t2 n2 ON n1.search_term <> n2.search_term
AND n1.search_term % n2.search_term
ORDER BY sim DESC)
Any help greatly appreciated!
postgresql grouping similarity
postgresql grouping similarity
asked Nov 22 '18 at 18:09
user10670868user10670868
111
asked Nov 22 '18 at 18:09
user10670868user10670868
111
asked Nov 22 '18 at 18:09
user10670868user10670868
111
asked Nov 22 '18 at 18:09
user10670868user10670868
111
asked Nov 22 '18 at 18:09
user10670868user10670868
111
111
add a comment |
add a comment |
1 Answer
1
active
oldest
votes
The value 0.8 is not enough. because similarity in your example is 0.6 and more
Try this query
SELECT sim, ss, sum(countt)
FROM (
SELECT sim, '|'||string_agg(s1, '|')||'|' ss
FROM (
SELECT similarity(n1.search_term, n2.search_term) AS sim,
n1.search_term s1, n2.search_term s2
FROM t1 n1
JOIN t1 n2 ON n1.search_term <> n2.search_term
AND n1.search_term % n2.search_term
) t2
WHERE sim > 0.6
GROUP BY sim
) t3
LEFT JOIN t1 n3 ON ss like '%|'||n3.search_term||'|%'
GROUP BY ss, sim
ORDER BY sim DESC
Here my sample - http://sqlfiddle.com/#!17/1d705/35
answered Nov 22 '18 at 19:53
SaidolimSaidolim
339817
Hey, thanks for the help but that didn't really work as it you try to add more terms to your schema, they get lost. For example, try: ('electric scooer',3), ('elecric scooter',4), ('electric scooter',20), ('road bicycle',5), ('camera',15), ('5D camera',12), ('segway longboard',15), ('boosted longboard',15);
– user10670868
Nov 23 '18 at 15:48
@user10670868 you can check sqlfiddle.com/#!17/33cc1/5 link. There is data. But maybe, similarity value is low
– Saidolim
Nov 26 '18 at 0:37
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53436298%2fcounting-number-of-similar-strings-in-postgresql%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
The value 0.8 is not enough. because similarity in your example is 0.6 and more
Try this query
SELECT sim, ss, sum(countt)
FROM (
SELECT sim, '|'||string_agg(s1, '|')||'|' ss
FROM (
SELECT similarity(n1.search_term, n2.search_term) AS sim,
n1.search_term s1, n2.search_term s2
FROM t1 n1
JOIN t1 n2 ON n1.search_term <> n2.search_term
AND n1.search_term % n2.search_term
) t2
WHERE sim > 0.6
GROUP BY sim
) t3
LEFT JOIN t1 n3 ON ss like '%|'||n3.search_term||'|%'
GROUP BY ss, sim
ORDER BY sim DESC
Here my sample - http://sqlfiddle.com/#!17/1d705/35
answered Nov 22 '18 at 19:53
SaidolimSaidolim
339817
Hey, thanks for the help but that didn't really work as it you try to add more terms to your schema, they get lost. For example, try: ('electric scooer',3), ('elecric scooter',4), ('electric scooter',20), ('road bicycle',5), ('camera',15), ('5D camera',12), ('segway longboard',15), ('boosted longboard',15);
– user10670868
Nov 23 '18 at 15:48
@user10670868 you can check sqlfiddle.com/#!17/33cc1/5 link. There is data. But maybe, similarity value is low
– Saidolim
Nov 26 '18 at 0:37
add a comment |
The value 0.8 is not enough. because similarity in your example is 0.6 and more
Try this query
SELECT sim, ss, sum(countt)
FROM (
SELECT sim, '|'||string_agg(s1, '|')||'|' ss
FROM (
SELECT similarity(n1.search_term, n2.search_term) AS sim,
n1.search_term s1, n2.search_term s2
FROM t1 n1
JOIN t1 n2 ON n1.search_term <> n2.search_term
AND n1.search_term % n2.search_term
) t2
WHERE sim > 0.6
GROUP BY sim
) t3
LEFT JOIN t1 n3 ON ss like '%|'||n3.search_term||'|%'
GROUP BY ss, sim
ORDER BY sim DESC
Here my sample - http://sqlfiddle.com/#!17/1d705/35
answered Nov 22 '18 at 19:53
SaidolimSaidolim
339817
Hey, thanks for the help but that didn't really work as it you try to add more terms to your schema, they get lost. For example, try: ('electric scooer',3), ('elecric scooter',4), ('electric scooter',20), ('road bicycle',5), ('camera',15), ('5D camera',12), ('segway longboard',15), ('boosted longboard',15);
– user10670868
Nov 23 '18 at 15:48
@user10670868 you can check sqlfiddle.com/#!17/33cc1/5 link. There is data. But maybe, similarity value is low
– Saidolim
Nov 26 '18 at 0:37
add a comment |
The value 0.8 is not enough. because similarity in your example is 0.6 and more
Try this query
SELECT sim, ss, sum(countt)
FROM (
SELECT sim, '|'||string_agg(s1, '|')||'|' ss
FROM (
SELECT similarity(n1.search_term, n2.search_term) AS sim,
n1.search_term s1, n2.search_term s2
FROM t1 n1
JOIN t1 n2 ON n1.search_term <> n2.search_term
AND n1.search_term % n2.search_term
) t2
WHERE sim > 0.6
GROUP BY sim
) t3
LEFT JOIN t1 n3 ON ss like '%|'||n3.search_term||'|%'
GROUP BY ss, sim
ORDER BY sim DESC
Here my sample - http://sqlfiddle.com/#!17/1d705/35
answered Nov 22 '18 at 19:53
SaidolimSaidolim
339817
The value 0.8 is not enough. because similarity in your example is 0.6 and more
Try this query
SELECT sim, ss, sum(countt)
FROM (
SELECT sim, '|'||string_agg(s1, '|')||'|' ss
FROM (
SELECT similarity(n1.search_term, n2.search_term) AS sim,
n1.search_term s1, n2.search_term s2
FROM t1 n1
JOIN t1 n2 ON n1.search_term <> n2.search_term
AND n1.search_term % n2.search_term
) t2
WHERE sim > 0.6
GROUP BY sim
) t3
LEFT JOIN t1 n3 ON ss like '%|'||n3.search_term||'|%'
GROUP BY ss, sim
ORDER BY sim DESC
Here my sample - http://sqlfiddle.com/#!17/1d705/35
answered Nov 22 '18 at 19:53
SaidolimSaidolim
339817
edited Nov 22 '18 at 19:59
answered Nov 22 '18 at 19:53
SaidolimSaidolim
339817
answered Nov 22 '18 at 19:53
SaidolimSaidolim
339817
answered Nov 22 '18 at 19:53
SaidolimSaidolim
339817
339817
Hey, thanks for the help but that didn't really work as it you try to add more terms to your schema, they get lost. For example, try: ('electric scooer',3), ('elecric scooter',4), ('electric scooter',20), ('road bicycle',5), ('camera',15), ('5D camera',12), ('segway longboard',15), ('boosted longboard',15);
– user10670868
Nov 23 '18 at 15:48
@user10670868 you can check sqlfiddle.com/#!17/33cc1/5 link. There is data. But maybe, similarity value is low
– Saidolim
Nov 26 '18 at 0:37
add a comment |
Hey, thanks for the help but that didn't really work as it you try to add more terms to your schema, they get lost. For example, try: ('electric scooer',3), ('elecric scooter',4), ('electric scooter',20), ('road bicycle',5), ('camera',15), ('5D camera',12), ('segway longboard',15), ('boosted longboard',15);
– user10670868
Nov 23 '18 at 15:48
@user10670868 you can check sqlfiddle.com/#!17/33cc1/5 link. There is data. But maybe, similarity value is low
– Saidolim
Nov 26 '18 at 0:37
Hey, thanks for the help but that didn't really work as it you try to add more terms to your schema, they get lost. For example, try: ('electric scooer',3), ('elecric scooter',4), ('electric scooter',20), ('road bicycle',5), ('camera',15), ('5D camera',12), ('segway longboard',15), ('boosted longboard',15);
– user10670868
Nov 23 '18 at 15:48
Hey, thanks for the help but that didn't really work as it you try to add more terms to your schema, they get lost. For example, try: ('electric scooer',3), ('elecric scooter',4), ('electric scooter',20), ('road bicycle',5), ('camera',15), ('5D camera',12), ('segway longboard',15), ('boosted longboard',15);
– user10670868
Nov 23 '18 at 15:48
@user10670868 you can check sqlfiddle.com/#!17/33cc1/5 link. There is data. But maybe, similarity value is low
– Saidolim
Nov 26 '18 at 0:37
@user10670868 you can check sqlfiddle.com/#!17/33cc1/5 link. There is data. But maybe, similarity value is low
– Saidolim
Nov 26 '18 at 0:37
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53436298%2fcounting-number-of-similar-strings-in-postgresql%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
