SQL-Query (with subquery, group and order by) optimization











up vote
0
down vote

favorite












coud you help me optimizing the following statement. It has a bad prerformance when dealing with huge amount of data (in my case 3Mio Messages and 25Mio MessageWorkItems).



Does anybody have any suggestions? Thank you in advance.



select distinct msg.id,  msgWorkItem_1.description
from message msg
left outer join message_work_item msgWorkItem_1 on msg.id=msgWorkItem_1.message_id
and ( msgWorkItem_1.id in (
select max(msgWorkItem_2.id)
from message_work_item msgWorkItem_2
inner join message_work_item_type msgWorkItem_Type on msgWorkItem_2.message_work_item_type_id=msgWorkItem_Type.id
where
msgWorkItem_2.creation_type= 'mobile'
and msgWorkItem_2.description is not null
and msgWorkItem_Type.code <> 'sent-to-app-manually'
-- Is it possible to avoid this correlation to the outer query ? )
and msgWorkItem_2.message_id = msg.id)
)
where msg.deactivation_time > ?
order by msgWorkItem_1.description asc









share|improve this question







New contributor




Bakahoui is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
















  • 1




    My suggestion is that you explain the logic of this query and/or give us a more minimal example to work with. Right now, your question is of the "why isn't this code working" variety.
    – Tim Biegeleisen
    2 days ago















up vote
0
down vote

favorite












coud you help me optimizing the following statement. It has a bad prerformance when dealing with huge amount of data (in my case 3Mio Messages and 25Mio MessageWorkItems).



Does anybody have any suggestions? Thank you in advance.



select distinct msg.id,  msgWorkItem_1.description
from message msg
left outer join message_work_item msgWorkItem_1 on msg.id=msgWorkItem_1.message_id
and ( msgWorkItem_1.id in (
select max(msgWorkItem_2.id)
from message_work_item msgWorkItem_2
inner join message_work_item_type msgWorkItem_Type on msgWorkItem_2.message_work_item_type_id=msgWorkItem_Type.id
where
msgWorkItem_2.creation_type= 'mobile'
and msgWorkItem_2.description is not null
and msgWorkItem_Type.code <> 'sent-to-app-manually'
-- Is it possible to avoid this correlation to the outer query ? )
and msgWorkItem_2.message_id = msg.id)
)
where msg.deactivation_time > ?
order by msgWorkItem_1.description asc









share|improve this question







New contributor




Bakahoui is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
















  • 1




    My suggestion is that you explain the logic of this query and/or give us a more minimal example to work with. Right now, your question is of the "why isn't this code working" variety.
    – Tim Biegeleisen
    2 days ago













up vote
0
down vote

favorite









up vote
0
down vote

favorite











coud you help me optimizing the following statement. It has a bad prerformance when dealing with huge amount of data (in my case 3Mio Messages and 25Mio MessageWorkItems).



Does anybody have any suggestions? Thank you in advance.



select distinct msg.id,  msgWorkItem_1.description
from message msg
left outer join message_work_item msgWorkItem_1 on msg.id=msgWorkItem_1.message_id
and ( msgWorkItem_1.id in (
select max(msgWorkItem_2.id)
from message_work_item msgWorkItem_2
inner join message_work_item_type msgWorkItem_Type on msgWorkItem_2.message_work_item_type_id=msgWorkItem_Type.id
where
msgWorkItem_2.creation_type= 'mobile'
and msgWorkItem_2.description is not null
and msgWorkItem_Type.code <> 'sent-to-app-manually'
-- Is it possible to avoid this correlation to the outer query ? )
and msgWorkItem_2.message_id = msg.id)
)
where msg.deactivation_time > ?
order by msgWorkItem_1.description asc









share|improve this question







New contributor




Bakahoui is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.











coud you help me optimizing the following statement. It has a bad prerformance when dealing with huge amount of data (in my case 3Mio Messages and 25Mio MessageWorkItems).



Does anybody have any suggestions? Thank you in advance.



select distinct msg.id,  msgWorkItem_1.description
from message msg
left outer join message_work_item msgWorkItem_1 on msg.id=msgWorkItem_1.message_id
and ( msgWorkItem_1.id in (
select max(msgWorkItem_2.id)
from message_work_item msgWorkItem_2
inner join message_work_item_type msgWorkItem_Type on msgWorkItem_2.message_work_item_type_id=msgWorkItem_Type.id
where
msgWorkItem_2.creation_type= 'mobile'
and msgWorkItem_2.description is not null
and msgWorkItem_Type.code <> 'sent-to-app-manually'
-- Is it possible to avoid this correlation to the outer query ? )
and msgWorkItem_2.message_id = msg.id)
)
where msg.deactivation_time > ?
order by msgWorkItem_1.description asc






sql query-optimization






share|improve this question







New contributor




Bakahoui is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.











share|improve this question







New contributor




Bakahoui is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.









share|improve this question




share|improve this question






New contributor




Bakahoui is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.









asked 2 days ago









Bakahoui

1




1




New contributor




Bakahoui is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.





New contributor





Bakahoui is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.






Bakahoui is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.








  • 1




    My suggestion is that you explain the logic of this query and/or give us a more minimal example to work with. Right now, your question is of the "why isn't this code working" variety.
    – Tim Biegeleisen
    2 days ago














  • 1




    My suggestion is that you explain the logic of this query and/or give us a more minimal example to work with. Right now, your question is of the "why isn't this code working" variety.
    – Tim Biegeleisen
    2 days ago








1




1




My suggestion is that you explain the logic of this query and/or give us a more minimal example to work with. Right now, your question is of the "why isn't this code working" variety.
– Tim Biegeleisen
2 days ago




My suggestion is that you explain the logic of this query and/or give us a more minimal example to work with. Right now, your question is of the "why isn't this code working" variety.
– Tim Biegeleisen
2 days ago












1 Answer
1






active

oldest

votes

















up vote
0
down vote













STEP 1 : Lay out the query so I have any hope of reading it



SELECT
DISTINCT
msg.id,
msgWorkItem_1.description
FROM
message msg
LEFT OUTER JOIN
message_work_item AS msgWorkItem_1
ON msgWorkItem_1.message_id = msg.id
AND msgWorkItem_1.id =
(
SELECT
MAX(msgWorkItem_2.id)
FROM
message_work_item AS msgWorkItem_2
INNER JOIN
message_work_item_type AS msgWorkItem_Type
ON msgWorkItem_2.message_work_item_type_id=msgWorkItem_Type.id
WHERE
msgWorkItem_2.creation_type= 'mobile'
AND msgWorkItem_2.description IS NOT NULL
AND msgWorkItem_Type.code <> 'sent-to-app-manually'
-- Is it possible to avoid this correlation to the outer query ?
AND msgWorkItem_2.message_id = msg.id
)
WHERE
msg.deactivation_time > ?
ORDER BY
msgWorkItem_1.description ASC


STEP 2 : rewrite using analytic functions instead of MAX()



SELECT
DISTINCT
message.id,
message_work_item_sorted.description
FROM
message
LEFT OUTER JOIN
(
SELECT
message_work_item.message_id,
message_work_item.description,
ROW_NUMBER() OVER (PARTITION BY message_work_item.message_id
ORDER BY message_work_item.id DESC
)
AS row_ordinal
FROM
message_work_item
INNER JOIN
message_work_item_type
ON message_work_item.message_work_item_type_id = message_work_item_type.id
WHERE
message_work_item.creation_type= 'mobile'
AND message_work_item.description IS NOT NULL
AND message_work_item_type.code <> 'sent-to-app-manually'
)
message_work_item_sorted
ON message_work_item_sorted.message_id = message.id
AND message_work_item_sorted.row_ordinal = 1
WHERE
message.deactivation_time > ?
ORDER BY
message_work_item_sorted.description ASC


With more information we could probably help further, but as you gave no definition of the tables, constraints, or business logic, this is just a re-write of what you're already implemented.



For example, I strongly doubt you need the DISTINCT (provided that the id columns in your tables are unique).






share|improve this answer





















    Your Answer






    StackExchange.ifUsing("editor", function () {
    StackExchange.using("externalEditor", function () {
    StackExchange.using("snippets", function () {
    StackExchange.snippets.init();
    });
    });
    }, "code-snippets");

    StackExchange.ready(function() {
    var channelOptions = {
    tags: "".split(" "),
    id: "1"
    };
    initTagRenderer("".split(" "), "".split(" "), channelOptions);

    StackExchange.using("externalEditor", function() {
    // Have to fire editor after snippets, if snippets enabled
    if (StackExchange.settings.snippets.snippetsEnabled) {
    StackExchange.using("snippets", function() {
    createEditor();
    });
    }
    else {
    createEditor();
    }
    });

    function createEditor() {
    StackExchange.prepareEditor({
    heartbeatType: 'answer',
    convertImagesToLinks: true,
    noModals: true,
    showLowRepImageUploadWarning: true,
    reputationToPostImages: 10,
    bindNavPrevention: true,
    postfix: "",
    imageUploader: {
    brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
    contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
    allowUrls: true
    },
    onDemand: true,
    discardSelector: ".discard-answer"
    ,immediatelyShowMarkdownHelp:true
    });


    }
    });






    Bakahoui is a new contributor. Be nice, and check out our Code of Conduct.










     

    draft saved


    draft discarded


















    StackExchange.ready(
    function () {
    StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53373041%2fsql-query-with-subquery-group-and-order-by-optimization%23new-answer', 'question_page');
    }
    );

    Post as a guest















    Required, but never shown

























    1 Answer
    1






    active

    oldest

    votes








    1 Answer
    1






    active

    oldest

    votes









    active

    oldest

    votes






    active

    oldest

    votes








    up vote
    0
    down vote













    STEP 1 : Lay out the query so I have any hope of reading it



    SELECT
    DISTINCT
    msg.id,
    msgWorkItem_1.description
    FROM
    message msg
    LEFT OUTER JOIN
    message_work_item AS msgWorkItem_1
    ON msgWorkItem_1.message_id = msg.id
    AND msgWorkItem_1.id =
    (
    SELECT
    MAX(msgWorkItem_2.id)
    FROM
    message_work_item AS msgWorkItem_2
    INNER JOIN
    message_work_item_type AS msgWorkItem_Type
    ON msgWorkItem_2.message_work_item_type_id=msgWorkItem_Type.id
    WHERE
    msgWorkItem_2.creation_type= 'mobile'
    AND msgWorkItem_2.description IS NOT NULL
    AND msgWorkItem_Type.code <> 'sent-to-app-manually'
    -- Is it possible to avoid this correlation to the outer query ?
    AND msgWorkItem_2.message_id = msg.id
    )
    WHERE
    msg.deactivation_time > ?
    ORDER BY
    msgWorkItem_1.description ASC


    STEP 2 : rewrite using analytic functions instead of MAX()



    SELECT
    DISTINCT
    message.id,
    message_work_item_sorted.description
    FROM
    message
    LEFT OUTER JOIN
    (
    SELECT
    message_work_item.message_id,
    message_work_item.description,
    ROW_NUMBER() OVER (PARTITION BY message_work_item.message_id
    ORDER BY message_work_item.id DESC
    )
    AS row_ordinal
    FROM
    message_work_item
    INNER JOIN
    message_work_item_type
    ON message_work_item.message_work_item_type_id = message_work_item_type.id
    WHERE
    message_work_item.creation_type= 'mobile'
    AND message_work_item.description IS NOT NULL
    AND message_work_item_type.code <> 'sent-to-app-manually'
    )
    message_work_item_sorted
    ON message_work_item_sorted.message_id = message.id
    AND message_work_item_sorted.row_ordinal = 1
    WHERE
    message.deactivation_time > ?
    ORDER BY
    message_work_item_sorted.description ASC


    With more information we could probably help further, but as you gave no definition of the tables, constraints, or business logic, this is just a re-write of what you're already implemented.



    For example, I strongly doubt you need the DISTINCT (provided that the id columns in your tables are unique).






    share|improve this answer

























      up vote
      0
      down vote













      STEP 1 : Lay out the query so I have any hope of reading it



      SELECT
      DISTINCT
      msg.id,
      msgWorkItem_1.description
      FROM
      message msg
      LEFT OUTER JOIN
      message_work_item AS msgWorkItem_1
      ON msgWorkItem_1.message_id = msg.id
      AND msgWorkItem_1.id =
      (
      SELECT
      MAX(msgWorkItem_2.id)
      FROM
      message_work_item AS msgWorkItem_2
      INNER JOIN
      message_work_item_type AS msgWorkItem_Type
      ON msgWorkItem_2.message_work_item_type_id=msgWorkItem_Type.id
      WHERE
      msgWorkItem_2.creation_type= 'mobile'
      AND msgWorkItem_2.description IS NOT NULL
      AND msgWorkItem_Type.code <> 'sent-to-app-manually'
      -- Is it possible to avoid this correlation to the outer query ?
      AND msgWorkItem_2.message_id = msg.id
      )
      WHERE
      msg.deactivation_time > ?
      ORDER BY
      msgWorkItem_1.description ASC


      STEP 2 : rewrite using analytic functions instead of MAX()



      SELECT
      DISTINCT
      message.id,
      message_work_item_sorted.description
      FROM
      message
      LEFT OUTER JOIN
      (
      SELECT
      message_work_item.message_id,
      message_work_item.description,
      ROW_NUMBER() OVER (PARTITION BY message_work_item.message_id
      ORDER BY message_work_item.id DESC
      )
      AS row_ordinal
      FROM
      message_work_item
      INNER JOIN
      message_work_item_type
      ON message_work_item.message_work_item_type_id = message_work_item_type.id
      WHERE
      message_work_item.creation_type= 'mobile'
      AND message_work_item.description IS NOT NULL
      AND message_work_item_type.code <> 'sent-to-app-manually'
      )
      message_work_item_sorted
      ON message_work_item_sorted.message_id = message.id
      AND message_work_item_sorted.row_ordinal = 1
      WHERE
      message.deactivation_time > ?
      ORDER BY
      message_work_item_sorted.description ASC


      With more information we could probably help further, but as you gave no definition of the tables, constraints, or business logic, this is just a re-write of what you're already implemented.



      For example, I strongly doubt you need the DISTINCT (provided that the id columns in your tables are unique).






      share|improve this answer























        up vote
        0
        down vote










        up vote
        0
        down vote









        STEP 1 : Lay out the query so I have any hope of reading it



        SELECT
        DISTINCT
        msg.id,
        msgWorkItem_1.description
        FROM
        message msg
        LEFT OUTER JOIN
        message_work_item AS msgWorkItem_1
        ON msgWorkItem_1.message_id = msg.id
        AND msgWorkItem_1.id =
        (
        SELECT
        MAX(msgWorkItem_2.id)
        FROM
        message_work_item AS msgWorkItem_2
        INNER JOIN
        message_work_item_type AS msgWorkItem_Type
        ON msgWorkItem_2.message_work_item_type_id=msgWorkItem_Type.id
        WHERE
        msgWorkItem_2.creation_type= 'mobile'
        AND msgWorkItem_2.description IS NOT NULL
        AND msgWorkItem_Type.code <> 'sent-to-app-manually'
        -- Is it possible to avoid this correlation to the outer query ?
        AND msgWorkItem_2.message_id = msg.id
        )
        WHERE
        msg.deactivation_time > ?
        ORDER BY
        msgWorkItem_1.description ASC


        STEP 2 : rewrite using analytic functions instead of MAX()



        SELECT
        DISTINCT
        message.id,
        message_work_item_sorted.description
        FROM
        message
        LEFT OUTER JOIN
        (
        SELECT
        message_work_item.message_id,
        message_work_item.description,
        ROW_NUMBER() OVER (PARTITION BY message_work_item.message_id
        ORDER BY message_work_item.id DESC
        )
        AS row_ordinal
        FROM
        message_work_item
        INNER JOIN
        message_work_item_type
        ON message_work_item.message_work_item_type_id = message_work_item_type.id
        WHERE
        message_work_item.creation_type= 'mobile'
        AND message_work_item.description IS NOT NULL
        AND message_work_item_type.code <> 'sent-to-app-manually'
        )
        message_work_item_sorted
        ON message_work_item_sorted.message_id = message.id
        AND message_work_item_sorted.row_ordinal = 1
        WHERE
        message.deactivation_time > ?
        ORDER BY
        message_work_item_sorted.description ASC


        With more information we could probably help further, but as you gave no definition of the tables, constraints, or business logic, this is just a re-write of what you're already implemented.



        For example, I strongly doubt you need the DISTINCT (provided that the id columns in your tables are unique).






        share|improve this answer












        STEP 1 : Lay out the query so I have any hope of reading it



        SELECT
        DISTINCT
        msg.id,
        msgWorkItem_1.description
        FROM
        message msg
        LEFT OUTER JOIN
        message_work_item AS msgWorkItem_1
        ON msgWorkItem_1.message_id = msg.id
        AND msgWorkItem_1.id =
        (
        SELECT
        MAX(msgWorkItem_2.id)
        FROM
        message_work_item AS msgWorkItem_2
        INNER JOIN
        message_work_item_type AS msgWorkItem_Type
        ON msgWorkItem_2.message_work_item_type_id=msgWorkItem_Type.id
        WHERE
        msgWorkItem_2.creation_type= 'mobile'
        AND msgWorkItem_2.description IS NOT NULL
        AND msgWorkItem_Type.code <> 'sent-to-app-manually'
        -- Is it possible to avoid this correlation to the outer query ?
        AND msgWorkItem_2.message_id = msg.id
        )
        WHERE
        msg.deactivation_time > ?
        ORDER BY
        msgWorkItem_1.description ASC


        STEP 2 : rewrite using analytic functions instead of MAX()



        SELECT
        DISTINCT
        message.id,
        message_work_item_sorted.description
        FROM
        message
        LEFT OUTER JOIN
        (
        SELECT
        message_work_item.message_id,
        message_work_item.description,
        ROW_NUMBER() OVER (PARTITION BY message_work_item.message_id
        ORDER BY message_work_item.id DESC
        )
        AS row_ordinal
        FROM
        message_work_item
        INNER JOIN
        message_work_item_type
        ON message_work_item.message_work_item_type_id = message_work_item_type.id
        WHERE
        message_work_item.creation_type= 'mobile'
        AND message_work_item.description IS NOT NULL
        AND message_work_item_type.code <> 'sent-to-app-manually'
        )
        message_work_item_sorted
        ON message_work_item_sorted.message_id = message.id
        AND message_work_item_sorted.row_ordinal = 1
        WHERE
        message.deactivation_time > ?
        ORDER BY
        message_work_item_sorted.description ASC


        With more information we could probably help further, but as you gave no definition of the tables, constraints, or business logic, this is just a re-write of what you're already implemented.



        For example, I strongly doubt you need the DISTINCT (provided that the id columns in your tables are unique).







        share|improve this answer












        share|improve this answer



        share|improve this answer










        answered 2 days ago









        MatBailie

        58.3k1373110




        58.3k1373110






















            Bakahoui is a new contributor. Be nice, and check out our Code of Conduct.










             

            draft saved


            draft discarded


















            Bakahoui is a new contributor. Be nice, and check out our Code of Conduct.













            Bakahoui is a new contributor. Be nice, and check out our Code of Conduct.












            Bakahoui is a new contributor. Be nice, and check out our Code of Conduct.















             


            draft saved


            draft discarded














            StackExchange.ready(
            function () {
            StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53373041%2fsql-query-with-subquery-group-and-order-by-optimization%23new-answer', 'question_page');
            }
            );

            Post as a guest















            Required, but never shown





















































            Required, but never shown














            Required, but never shown












            Required, but never shown







            Required, but never shown

































            Required, but never shown














            Required, but never shown












            Required, but never shown







            Required, but never shown







            Popular posts from this blog

            Wiesbaden

            Marschland

            Dieringhausen