$expr arrayElementAt not working in aggregation for embedded document











up vote
1
down vote

favorite












I am doing mongo db aggregation like



$cursor = $this->collection->aggregate(
array(
array(
'$project' => array(
'FullName' => array('$concat' => array('$first_name', ' ', '$middle_name', ' ', '$last_name')),
'FirstMiddle' => array('$concat' => array('$first_name', ' ', '$middle_name')),
'FirstLast' => array('$concat' => array('$first_name', ' ', '$last_name')),
'FirstName' => array('$concat' => array('$first_name')),
'MiddleName' => array('$concat' => array('$middle_name')),
'LastName' => array('$concat' => array('$last_name')),
'Student' => '$$ROOT'
)
),
array(
'$match' =>
array(
'$or' => array(
array("FullName" => new MongoDBBSONRegex($arg, 'i')),
array("FirstLast" => new MongoDBBSONRegex($arg, 'i')),
array("FirstMiddle" => new MongoDBBSONRegex($arg, 'i')),
array("FirstName" => new MongoDBBSONRegex($arg, 'i')),
array("MiddleName" => new MongoDBBSONRegex($arg, 'i')),
array("LastName" => new MongoDBBSONRegex($arg, 'i')),
array("Student.registration_temp_perm_no" => $arg),
'$expr' => array(
'$eq'=> array(
array('$arrayElemAt' => array('$allotment_details.room_id', -1)), $this->RoomId)),
),
// "Student.assigned_keys" => ['$exists' => false],
"Student.schoolId" => new MongoDBBSONObjectID($this->SchoolId)
)
)
)
);


I have collection which contains data like



"first_name": "John",
"middle_name": "",
"last_name": "Mayor",
"allotment_details": [
{
"allotment_id": "ff666d55-2fcc-79b2-e4da-e165939555bb",
"room_id": "5be2d9aad2ccda0fdc006a65",
"bay_id": ObjectId("5be2d9aad2ccda0fdc006a61"),
...
}


The above code is working for fine concatenation of three names types and searching all the data passed in $arg. Note that I have added array('$arrayElemAt' => array('$allotment_details.room_id', -1)), $this->RoomId)) in order to fetch students based on concatenation of names and those students should be fetched on the basis of $this->RoomId.



The above code is not fetching students assigned to a room but is fetching all the students based on concatenation of names.
Please help !!!










share|improve this question




















  • 1




    allotment_details is not included in your initial $project, so the field is not there. $project and $group only return the fields you "explicitly" name. You should not be doing this anyway. If you need to match the "last" array element, then you should be "reversing your array" in storage instead. Then simply match on the regular query 'allotment_details.0.room_id' since your "reversed" array last the newest added item "first".
    – Neil Lunn
    Nov 13 at 10:04






  • 1




    At any rate $match should pretty much always be the very first aggreation pipeline stage. Also the query is not really smart in general, and you really should be using $or instead of concatenating strings.
    – Neil Lunn
    Nov 13 at 10:06










  • can you please provide code ....
    – Nisa Nisa
    Nov 13 at 10:08















up vote
1
down vote

favorite












I am doing mongo db aggregation like



$cursor = $this->collection->aggregate(
array(
array(
'$project' => array(
'FullName' => array('$concat' => array('$first_name', ' ', '$middle_name', ' ', '$last_name')),
'FirstMiddle' => array('$concat' => array('$first_name', ' ', '$middle_name')),
'FirstLast' => array('$concat' => array('$first_name', ' ', '$last_name')),
'FirstName' => array('$concat' => array('$first_name')),
'MiddleName' => array('$concat' => array('$middle_name')),
'LastName' => array('$concat' => array('$last_name')),
'Student' => '$$ROOT'
)
),
array(
'$match' =>
array(
'$or' => array(
array("FullName" => new MongoDBBSONRegex($arg, 'i')),
array("FirstLast" => new MongoDBBSONRegex($arg, 'i')),
array("FirstMiddle" => new MongoDBBSONRegex($arg, 'i')),
array("FirstName" => new MongoDBBSONRegex($arg, 'i')),
array("MiddleName" => new MongoDBBSONRegex($arg, 'i')),
array("LastName" => new MongoDBBSONRegex($arg, 'i')),
array("Student.registration_temp_perm_no" => $arg),
'$expr' => array(
'$eq'=> array(
array('$arrayElemAt' => array('$allotment_details.room_id', -1)), $this->RoomId)),
),
// "Student.assigned_keys" => ['$exists' => false],
"Student.schoolId" => new MongoDBBSONObjectID($this->SchoolId)
)
)
)
);


I have collection which contains data like



"first_name": "John",
"middle_name": "",
"last_name": "Mayor",
"allotment_details": [
{
"allotment_id": "ff666d55-2fcc-79b2-e4da-e165939555bb",
"room_id": "5be2d9aad2ccda0fdc006a65",
"bay_id": ObjectId("5be2d9aad2ccda0fdc006a61"),
...
}


The above code is working for fine concatenation of three names types and searching all the data passed in $arg. Note that I have added array('$arrayElemAt' => array('$allotment_details.room_id', -1)), $this->RoomId)) in order to fetch students based on concatenation of names and those students should be fetched on the basis of $this->RoomId.



The above code is not fetching students assigned to a room but is fetching all the students based on concatenation of names.
Please help !!!










share|improve this question




















  • 1




    allotment_details is not included in your initial $project, so the field is not there. $project and $group only return the fields you "explicitly" name. You should not be doing this anyway. If you need to match the "last" array element, then you should be "reversing your array" in storage instead. Then simply match on the regular query 'allotment_details.0.room_id' since your "reversed" array last the newest added item "first".
    – Neil Lunn
    Nov 13 at 10:04






  • 1




    At any rate $match should pretty much always be the very first aggreation pipeline stage. Also the query is not really smart in general, and you really should be using $or instead of concatenating strings.
    – Neil Lunn
    Nov 13 at 10:06










  • can you please provide code ....
    – Nisa Nisa
    Nov 13 at 10:08













up vote
1
down vote

favorite









up vote
1
down vote

favorite











I am doing mongo db aggregation like



$cursor = $this->collection->aggregate(
array(
array(
'$project' => array(
'FullName' => array('$concat' => array('$first_name', ' ', '$middle_name', ' ', '$last_name')),
'FirstMiddle' => array('$concat' => array('$first_name', ' ', '$middle_name')),
'FirstLast' => array('$concat' => array('$first_name', ' ', '$last_name')),
'FirstName' => array('$concat' => array('$first_name')),
'MiddleName' => array('$concat' => array('$middle_name')),
'LastName' => array('$concat' => array('$last_name')),
'Student' => '$$ROOT'
)
),
array(
'$match' =>
array(
'$or' => array(
array("FullName" => new MongoDBBSONRegex($arg, 'i')),
array("FirstLast" => new MongoDBBSONRegex($arg, 'i')),
array("FirstMiddle" => new MongoDBBSONRegex($arg, 'i')),
array("FirstName" => new MongoDBBSONRegex($arg, 'i')),
array("MiddleName" => new MongoDBBSONRegex($arg, 'i')),
array("LastName" => new MongoDBBSONRegex($arg, 'i')),
array("Student.registration_temp_perm_no" => $arg),
'$expr' => array(
'$eq'=> array(
array('$arrayElemAt' => array('$allotment_details.room_id', -1)), $this->RoomId)),
),
// "Student.assigned_keys" => ['$exists' => false],
"Student.schoolId" => new MongoDBBSONObjectID($this->SchoolId)
)
)
)
);


I have collection which contains data like



"first_name": "John",
"middle_name": "",
"last_name": "Mayor",
"allotment_details": [
{
"allotment_id": "ff666d55-2fcc-79b2-e4da-e165939555bb",
"room_id": "5be2d9aad2ccda0fdc006a65",
"bay_id": ObjectId("5be2d9aad2ccda0fdc006a61"),
...
}


The above code is working for fine concatenation of three names types and searching all the data passed in $arg. Note that I have added array('$arrayElemAt' => array('$allotment_details.room_id', -1)), $this->RoomId)) in order to fetch students based on concatenation of names and those students should be fetched on the basis of $this->RoomId.



The above code is not fetching students assigned to a room but is fetching all the students based on concatenation of names.
Please help !!!










share|improve this question















I am doing mongo db aggregation like



$cursor = $this->collection->aggregate(
array(
array(
'$project' => array(
'FullName' => array('$concat' => array('$first_name', ' ', '$middle_name', ' ', '$last_name')),
'FirstMiddle' => array('$concat' => array('$first_name', ' ', '$middle_name')),
'FirstLast' => array('$concat' => array('$first_name', ' ', '$last_name')),
'FirstName' => array('$concat' => array('$first_name')),
'MiddleName' => array('$concat' => array('$middle_name')),
'LastName' => array('$concat' => array('$last_name')),
'Student' => '$$ROOT'
)
),
array(
'$match' =>
array(
'$or' => array(
array("FullName" => new MongoDBBSONRegex($arg, 'i')),
array("FirstLast" => new MongoDBBSONRegex($arg, 'i')),
array("FirstMiddle" => new MongoDBBSONRegex($arg, 'i')),
array("FirstName" => new MongoDBBSONRegex($arg, 'i')),
array("MiddleName" => new MongoDBBSONRegex($arg, 'i')),
array("LastName" => new MongoDBBSONRegex($arg, 'i')),
array("Student.registration_temp_perm_no" => $arg),
'$expr' => array(
'$eq'=> array(
array('$arrayElemAt' => array('$allotment_details.room_id', -1)), $this->RoomId)),
),
// "Student.assigned_keys" => ['$exists' => false],
"Student.schoolId" => new MongoDBBSONObjectID($this->SchoolId)
)
)
)
);


I have collection which contains data like



"first_name": "John",
"middle_name": "",
"last_name": "Mayor",
"allotment_details": [
{
"allotment_id": "ff666d55-2fcc-79b2-e4da-e165939555bb",
"room_id": "5be2d9aad2ccda0fdc006a65",
"bay_id": ObjectId("5be2d9aad2ccda0fdc006a61"),
...
}


The above code is working for fine concatenation of three names types and searching all the data passed in $arg. Note that I have added array('$arrayElemAt' => array('$allotment_details.room_id', -1)), $this->RoomId)) in order to fetch students based on concatenation of names and those students should be fetched on the basis of $this->RoomId.



The above code is not fetching students assigned to a room but is fetching all the students based on concatenation of names.
Please help !!!







mongodb mongodb-query aggregation-framework mongodb-php php-mongodb






share|improve this question















share|improve this question













share|improve this question




share|improve this question








edited Nov 13 at 10:12









Raghbendra Nayak

1,14711031




1,14711031










asked Nov 13 at 9:56









Nisa Nisa

186




186








  • 1




    allotment_details is not included in your initial $project, so the field is not there. $project and $group only return the fields you "explicitly" name. You should not be doing this anyway. If you need to match the "last" array element, then you should be "reversing your array" in storage instead. Then simply match on the regular query 'allotment_details.0.room_id' since your "reversed" array last the newest added item "first".
    – Neil Lunn
    Nov 13 at 10:04






  • 1




    At any rate $match should pretty much always be the very first aggreation pipeline stage. Also the query is not really smart in general, and you really should be using $or instead of concatenating strings.
    – Neil Lunn
    Nov 13 at 10:06










  • can you please provide code ....
    – Nisa Nisa
    Nov 13 at 10:08














  • 1




    allotment_details is not included in your initial $project, so the field is not there. $project and $group only return the fields you "explicitly" name. You should not be doing this anyway. If you need to match the "last" array element, then you should be "reversing your array" in storage instead. Then simply match on the regular query 'allotment_details.0.room_id' since your "reversed" array last the newest added item "first".
    – Neil Lunn
    Nov 13 at 10:04






  • 1




    At any rate $match should pretty much always be the very first aggreation pipeline stage. Also the query is not really smart in general, and you really should be using $or instead of concatenating strings.
    – Neil Lunn
    Nov 13 at 10:06










  • can you please provide code ....
    – Nisa Nisa
    Nov 13 at 10:08








1




1




allotment_details is not included in your initial $project, so the field is not there. $project and $group only return the fields you "explicitly" name. You should not be doing this anyway. If you need to match the "last" array element, then you should be "reversing your array" in storage instead. Then simply match on the regular query 'allotment_details.0.room_id' since your "reversed" array last the newest added item "first".
– Neil Lunn
Nov 13 at 10:04




allotment_details is not included in your initial $project, so the field is not there. $project and $group only return the fields you "explicitly" name. You should not be doing this anyway. If you need to match the "last" array element, then you should be "reversing your array" in storage instead. Then simply match on the regular query 'allotment_details.0.room_id' since your "reversed" array last the newest added item "first".
– Neil Lunn
Nov 13 at 10:04




1




1




At any rate $match should pretty much always be the very first aggreation pipeline stage. Also the query is not really smart in general, and you really should be using $or instead of concatenating strings.
– Neil Lunn
Nov 13 at 10:06




At any rate $match should pretty much always be the very first aggreation pipeline stage. Also the query is not really smart in general, and you really should be using $or instead of concatenating strings.
– Neil Lunn
Nov 13 at 10:06












can you please provide code ....
– Nisa Nisa
Nov 13 at 10:08




can you please provide code ....
– Nisa Nisa
Nov 13 at 10:08












1 Answer
1






active

oldest

votes

















up vote
0
down vote



accepted










Quick fix



Your "pipeline" does not work here primarily because your initial $project lacks the field you want to use an a later stage. The "quick fix" is therefore basically to include that field in the "projected" document, since that's how aggregation pipeline stages work:



array(
array(
'$project' => array(
'FullName' => array('$concat' => array('$first_name', ' ', '$middle_name', ' ', '$last_name')),
'FirstMiddle' => array('$concat' => array('$first_name', ' ', '$middle_name')),
'FirstLast' => array('$concat' => array('$first_name', ' ', '$last_name')),
'FirstName' => array('$concat' => array('$first_name')),
'MiddleName' => array('$concat' => array('$middle_name')),
'LastName' => array('$concat' => array('$last_name')),
'Student' => '$$ROOT',
'allotment_details' => 1 # that's the change
)
),


Or even since you used $$ROOT for Student anyway, simply qualify the field under that path:



'$expr' => array(
'$eq'=> array(
array('$arrayElemAt' => array('$Student.allotment_details.room_id', -1)),
$this->RoomId
)
),


however I would strongly* implore that you do NOT do that.



The whole concept of "concatenating strings" in order to do a later $match on the content is a really bad idea since it means the whole collection gets rewritten in the pipeline before any "filtering" actually gets done.



Likewise looking to match on the "last" array element is also an issue. A far better approach is to instead actually add "new items" to the "beginning" of the array, instead of the "end". This is actually what the $position or possibly even the $sort modifiers to $push do for you, by changing where items get added or the sorted order of items respectively.



Changing the Array to "newest first"



This takes a little work by changing the way you store things, but the benefits are greatly improved speed of such queries like you want without needing an evaluated $expr argument.



The core concepts are to "pre-pend" new array items with syntax like:



$this->collection->updateOne(
$query,
[ '$push' => [ 'allotment_details' => [ '$each' => $allotments, '$position' => 0 ] ] ]
)


Where $alloments must be an array as required by $each and $position is used to 0 in order to add the new array item "first".



Alternately if you actually have something like created_date as a property within each of the objects in the array, then you "could" use something like $sort as a modifier instead.



$this->collection->updateOne(
$query,
[ '$push' => [
'allotment_details' => [ '$each' => $allotments, '$sort' => [ 'created_date' => -1 ] ]
]]
)


It really depends on whether your "query" and other access requirements rely on "last added" or "latest date", and then also typically if you intend to possibly alter such a created_date or other "sort" property in a way which would effect the order of the array elements when "sorted".



The reason you do this is then matching the "latest" ( which is now the "first" ) item in the array simply becomes:



$this->collection->find([
'allotment_details.0.room_id': $this->RoomId
])


MongoDB allows the "first" array index to be specified with "Dot Notation", using the 0 index. What you cannot do is specify a "negative" index i.e:



$this->collection->find([
'allotment_details.-1.room_id': $this->RoomId # not allowed :(
])


That's the reason why you do the things shown above on "update" in order to "re-order" your array to the workable form.



Concatenation is Bad



The other main issue is the concatenation of strings. As already mentioned this creates unnecessary overhead just in order to do the matching you want. It's also "unnecessary" since you can complete avoid this using $or with the conditions on each of the fields as they exist already within the actual document:



 $this->collection->find([
'$or' => [
[ 'first_name' => new MongoDBBSONRegex($arg, 'i') ],
[ 'last_name' => new MongoDBBSONRegex($arg, 'i') ],
[ 'middle_name' => new MongoDBBSONRegex($arg, 'i') ],
[ 'registration_temp_perm_no' => $arg ]
],
'schoolId' => new MongoDBBSONObjectID($this->SchoolId),
'allotment_details.0.room_id': $this->RoomId
])


And of course whatever the "full" query conditions actually need to be, but you should be getting the basic idea.



Also if you are not actually looking for "partial words", then a "text search" defined over the fields with the "names". After creating the index that would be:



 $this->collection->find([
'$text' => [ '$search' => $arg ],
'schoolId' => new MongoDBBSONObjectID($this->SchoolId),
'allotment_details.0.room_id': $this->RoomId
])




Overall I would really recommend looking closely at all the other options rather than making one small change to your existing code. With a little careful re-structuring of how you store things and indeed "index" things, you get huge performance benefits that your extensive $concat "brute force" approach simply cannot deliver.




N.B Modern PHP Releases generally support as a much more brief representation of array(). It's a lot cleaner and far easier to read. So please use it.







share|improve this answer























  • Thanks for the effort u have put to answer my question
    – Nisa Nisa
    Nov 13 at 12:25











Your Answer






StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");

StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});

function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});


}
});














draft saved

draft discarded


















StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53278310%2fexpr-arrayelementat-not-working-in-aggregation-for-embedded-document%23new-answer', 'question_page');
}
);

Post as a guest















Required, but never shown

























1 Answer
1






active

oldest

votes








1 Answer
1






active

oldest

votes









active

oldest

votes






active

oldest

votes








up vote
0
down vote



accepted










Quick fix



Your "pipeline" does not work here primarily because your initial $project lacks the field you want to use an a later stage. The "quick fix" is therefore basically to include that field in the "projected" document, since that's how aggregation pipeline stages work:



array(
array(
'$project' => array(
'FullName' => array('$concat' => array('$first_name', ' ', '$middle_name', ' ', '$last_name')),
'FirstMiddle' => array('$concat' => array('$first_name', ' ', '$middle_name')),
'FirstLast' => array('$concat' => array('$first_name', ' ', '$last_name')),
'FirstName' => array('$concat' => array('$first_name')),
'MiddleName' => array('$concat' => array('$middle_name')),
'LastName' => array('$concat' => array('$last_name')),
'Student' => '$$ROOT',
'allotment_details' => 1 # that's the change
)
),


Or even since you used $$ROOT for Student anyway, simply qualify the field under that path:



'$expr' => array(
'$eq'=> array(
array('$arrayElemAt' => array('$Student.allotment_details.room_id', -1)),
$this->RoomId
)
),


however I would strongly* implore that you do NOT do that.



The whole concept of "concatenating strings" in order to do a later $match on the content is a really bad idea since it means the whole collection gets rewritten in the pipeline before any "filtering" actually gets done.



Likewise looking to match on the "last" array element is also an issue. A far better approach is to instead actually add "new items" to the "beginning" of the array, instead of the "end". This is actually what the $position or possibly even the $sort modifiers to $push do for you, by changing where items get added or the sorted order of items respectively.



Changing the Array to "newest first"



This takes a little work by changing the way you store things, but the benefits are greatly improved speed of such queries like you want without needing an evaluated $expr argument.



The core concepts are to "pre-pend" new array items with syntax like:



$this->collection->updateOne(
$query,
[ '$push' => [ 'allotment_details' => [ '$each' => $allotments, '$position' => 0 ] ] ]
)


Where $alloments must be an array as required by $each and $position is used to 0 in order to add the new array item "first".



Alternately if you actually have something like created_date as a property within each of the objects in the array, then you "could" use something like $sort as a modifier instead.



$this->collection->updateOne(
$query,
[ '$push' => [
'allotment_details' => [ '$each' => $allotments, '$sort' => [ 'created_date' => -1 ] ]
]]
)


It really depends on whether your "query" and other access requirements rely on "last added" or "latest date", and then also typically if you intend to possibly alter such a created_date or other "sort" property in a way which would effect the order of the array elements when "sorted".



The reason you do this is then matching the "latest" ( which is now the "first" ) item in the array simply becomes:



$this->collection->find([
'allotment_details.0.room_id': $this->RoomId
])


MongoDB allows the "first" array index to be specified with "Dot Notation", using the 0 index. What you cannot do is specify a "negative" index i.e:



$this->collection->find([
'allotment_details.-1.room_id': $this->RoomId # not allowed :(
])


That's the reason why you do the things shown above on "update" in order to "re-order" your array to the workable form.



Concatenation is Bad



The other main issue is the concatenation of strings. As already mentioned this creates unnecessary overhead just in order to do the matching you want. It's also "unnecessary" since you can complete avoid this using $or with the conditions on each of the fields as they exist already within the actual document:



 $this->collection->find([
'$or' => [
[ 'first_name' => new MongoDBBSONRegex($arg, 'i') ],
[ 'last_name' => new MongoDBBSONRegex($arg, 'i') ],
[ 'middle_name' => new MongoDBBSONRegex($arg, 'i') ],
[ 'registration_temp_perm_no' => $arg ]
],
'schoolId' => new MongoDBBSONObjectID($this->SchoolId),
'allotment_details.0.room_id': $this->RoomId
])


And of course whatever the "full" query conditions actually need to be, but you should be getting the basic idea.



Also if you are not actually looking for "partial words", then a "text search" defined over the fields with the "names". After creating the index that would be:



 $this->collection->find([
'$text' => [ '$search' => $arg ],
'schoolId' => new MongoDBBSONObjectID($this->SchoolId),
'allotment_details.0.room_id': $this->RoomId
])




Overall I would really recommend looking closely at all the other options rather than making one small change to your existing code. With a little careful re-structuring of how you store things and indeed "index" things, you get huge performance benefits that your extensive $concat "brute force" approach simply cannot deliver.




N.B Modern PHP Releases generally support as a much more brief representation of array(). It's a lot cleaner and far easier to read. So please use it.







share|improve this answer























  • Thanks for the effort u have put to answer my question
    – Nisa Nisa
    Nov 13 at 12:25















up vote
0
down vote



accepted










Quick fix



Your "pipeline" does not work here primarily because your initial $project lacks the field you want to use an a later stage. The "quick fix" is therefore basically to include that field in the "projected" document, since that's how aggregation pipeline stages work:



array(
array(
'$project' => array(
'FullName' => array('$concat' => array('$first_name', ' ', '$middle_name', ' ', '$last_name')),
'FirstMiddle' => array('$concat' => array('$first_name', ' ', '$middle_name')),
'FirstLast' => array('$concat' => array('$first_name', ' ', '$last_name')),
'FirstName' => array('$concat' => array('$first_name')),
'MiddleName' => array('$concat' => array('$middle_name')),
'LastName' => array('$concat' => array('$last_name')),
'Student' => '$$ROOT',
'allotment_details' => 1 # that's the change
)
),


Or even since you used $$ROOT for Student anyway, simply qualify the field under that path:



'$expr' => array(
'$eq'=> array(
array('$arrayElemAt' => array('$Student.allotment_details.room_id', -1)),
$this->RoomId
)
),


however I would strongly* implore that you do NOT do that.



The whole concept of "concatenating strings" in order to do a later $match on the content is a really bad idea since it means the whole collection gets rewritten in the pipeline before any "filtering" actually gets done.



Likewise looking to match on the "last" array element is also an issue. A far better approach is to instead actually add "new items" to the "beginning" of the array, instead of the "end". This is actually what the $position or possibly even the $sort modifiers to $push do for you, by changing where items get added or the sorted order of items respectively.



Changing the Array to "newest first"



This takes a little work by changing the way you store things, but the benefits are greatly improved speed of such queries like you want without needing an evaluated $expr argument.



The core concepts are to "pre-pend" new array items with syntax like:



$this->collection->updateOne(
$query,
[ '$push' => [ 'allotment_details' => [ '$each' => $allotments, '$position' => 0 ] ] ]
)


Where $alloments must be an array as required by $each and $position is used to 0 in order to add the new array item "first".



Alternately if you actually have something like created_date as a property within each of the objects in the array, then you "could" use something like $sort as a modifier instead.



$this->collection->updateOne(
$query,
[ '$push' => [
'allotment_details' => [ '$each' => $allotments, '$sort' => [ 'created_date' => -1 ] ]
]]
)


It really depends on whether your "query" and other access requirements rely on "last added" or "latest date", and then also typically if you intend to possibly alter such a created_date or other "sort" property in a way which would effect the order of the array elements when "sorted".



The reason you do this is then matching the "latest" ( which is now the "first" ) item in the array simply becomes:



$this->collection->find([
'allotment_details.0.room_id': $this->RoomId
])


MongoDB allows the "first" array index to be specified with "Dot Notation", using the 0 index. What you cannot do is specify a "negative" index i.e:



$this->collection->find([
'allotment_details.-1.room_id': $this->RoomId # not allowed :(
])


That's the reason why you do the things shown above on "update" in order to "re-order" your array to the workable form.



Concatenation is Bad



The other main issue is the concatenation of strings. As already mentioned this creates unnecessary overhead just in order to do the matching you want. It's also "unnecessary" since you can complete avoid this using $or with the conditions on each of the fields as they exist already within the actual document:



 $this->collection->find([
'$or' => [
[ 'first_name' => new MongoDBBSONRegex($arg, 'i') ],
[ 'last_name' => new MongoDBBSONRegex($arg, 'i') ],
[ 'middle_name' => new MongoDBBSONRegex($arg, 'i') ],
[ 'registration_temp_perm_no' => $arg ]
],
'schoolId' => new MongoDBBSONObjectID($this->SchoolId),
'allotment_details.0.room_id': $this->RoomId
])


And of course whatever the "full" query conditions actually need to be, but you should be getting the basic idea.



Also if you are not actually looking for "partial words", then a "text search" defined over the fields with the "names". After creating the index that would be:



 $this->collection->find([
'$text' => [ '$search' => $arg ],
'schoolId' => new MongoDBBSONObjectID($this->SchoolId),
'allotment_details.0.room_id': $this->RoomId
])




Overall I would really recommend looking closely at all the other options rather than making one small change to your existing code. With a little careful re-structuring of how you store things and indeed "index" things, you get huge performance benefits that your extensive $concat "brute force" approach simply cannot deliver.




N.B Modern PHP Releases generally support as a much more brief representation of array(). It's a lot cleaner and far easier to read. So please use it.







share|improve this answer























  • Thanks for the effort u have put to answer my question
    – Nisa Nisa
    Nov 13 at 12:25













up vote
0
down vote



accepted







up vote
0
down vote



accepted






Quick fix



Your "pipeline" does not work here primarily because your initial $project lacks the field you want to use an a later stage. The "quick fix" is therefore basically to include that field in the "projected" document, since that's how aggregation pipeline stages work:



array(
array(
'$project' => array(
'FullName' => array('$concat' => array('$first_name', ' ', '$middle_name', ' ', '$last_name')),
'FirstMiddle' => array('$concat' => array('$first_name', ' ', '$middle_name')),
'FirstLast' => array('$concat' => array('$first_name', ' ', '$last_name')),
'FirstName' => array('$concat' => array('$first_name')),
'MiddleName' => array('$concat' => array('$middle_name')),
'LastName' => array('$concat' => array('$last_name')),
'Student' => '$$ROOT',
'allotment_details' => 1 # that's the change
)
),


Or even since you used $$ROOT for Student anyway, simply qualify the field under that path:



'$expr' => array(
'$eq'=> array(
array('$arrayElemAt' => array('$Student.allotment_details.room_id', -1)),
$this->RoomId
)
),


however I would strongly* implore that you do NOT do that.



The whole concept of "concatenating strings" in order to do a later $match on the content is a really bad idea since it means the whole collection gets rewritten in the pipeline before any "filtering" actually gets done.



Likewise looking to match on the "last" array element is also an issue. A far better approach is to instead actually add "new items" to the "beginning" of the array, instead of the "end". This is actually what the $position or possibly even the $sort modifiers to $push do for you, by changing where items get added or the sorted order of items respectively.



Changing the Array to "newest first"



This takes a little work by changing the way you store things, but the benefits are greatly improved speed of such queries like you want without needing an evaluated $expr argument.



The core concepts are to "pre-pend" new array items with syntax like:



$this->collection->updateOne(
$query,
[ '$push' => [ 'allotment_details' => [ '$each' => $allotments, '$position' => 0 ] ] ]
)


Where $alloments must be an array as required by $each and $position is used to 0 in order to add the new array item "first".



Alternately if you actually have something like created_date as a property within each of the objects in the array, then you "could" use something like $sort as a modifier instead.



$this->collection->updateOne(
$query,
[ '$push' => [
'allotment_details' => [ '$each' => $allotments, '$sort' => [ 'created_date' => -1 ] ]
]]
)


It really depends on whether your "query" and other access requirements rely on "last added" or "latest date", and then also typically if you intend to possibly alter such a created_date or other "sort" property in a way which would effect the order of the array elements when "sorted".



The reason you do this is then matching the "latest" ( which is now the "first" ) item in the array simply becomes:



$this->collection->find([
'allotment_details.0.room_id': $this->RoomId
])


MongoDB allows the "first" array index to be specified with "Dot Notation", using the 0 index. What you cannot do is specify a "negative" index i.e:



$this->collection->find([
'allotment_details.-1.room_id': $this->RoomId # not allowed :(
])


That's the reason why you do the things shown above on "update" in order to "re-order" your array to the workable form.



Concatenation is Bad



The other main issue is the concatenation of strings. As already mentioned this creates unnecessary overhead just in order to do the matching you want. It's also "unnecessary" since you can complete avoid this using $or with the conditions on each of the fields as they exist already within the actual document:



 $this->collection->find([
'$or' => [
[ 'first_name' => new MongoDBBSONRegex($arg, 'i') ],
[ 'last_name' => new MongoDBBSONRegex($arg, 'i') ],
[ 'middle_name' => new MongoDBBSONRegex($arg, 'i') ],
[ 'registration_temp_perm_no' => $arg ]
],
'schoolId' => new MongoDBBSONObjectID($this->SchoolId),
'allotment_details.0.room_id': $this->RoomId
])


And of course whatever the "full" query conditions actually need to be, but you should be getting the basic idea.



Also if you are not actually looking for "partial words", then a "text search" defined over the fields with the "names". After creating the index that would be:



 $this->collection->find([
'$text' => [ '$search' => $arg ],
'schoolId' => new MongoDBBSONObjectID($this->SchoolId),
'allotment_details.0.room_id': $this->RoomId
])




Overall I would really recommend looking closely at all the other options rather than making one small change to your existing code. With a little careful re-structuring of how you store things and indeed "index" things, you get huge performance benefits that your extensive $concat "brute force" approach simply cannot deliver.




N.B Modern PHP Releases generally support as a much more brief representation of array(). It's a lot cleaner and far easier to read. So please use it.







share|improve this answer














Quick fix



Your "pipeline" does not work here primarily because your initial $project lacks the field you want to use an a later stage. The "quick fix" is therefore basically to include that field in the "projected" document, since that's how aggregation pipeline stages work:



array(
array(
'$project' => array(
'FullName' => array('$concat' => array('$first_name', ' ', '$middle_name', ' ', '$last_name')),
'FirstMiddle' => array('$concat' => array('$first_name', ' ', '$middle_name')),
'FirstLast' => array('$concat' => array('$first_name', ' ', '$last_name')),
'FirstName' => array('$concat' => array('$first_name')),
'MiddleName' => array('$concat' => array('$middle_name')),
'LastName' => array('$concat' => array('$last_name')),
'Student' => '$$ROOT',
'allotment_details' => 1 # that's the change
)
),


Or even since you used $$ROOT for Student anyway, simply qualify the field under that path:



'$expr' => array(
'$eq'=> array(
array('$arrayElemAt' => array('$Student.allotment_details.room_id', -1)),
$this->RoomId
)
),


however I would strongly* implore that you do NOT do that.



The whole concept of "concatenating strings" in order to do a later $match on the content is a really bad idea since it means the whole collection gets rewritten in the pipeline before any "filtering" actually gets done.



Likewise looking to match on the "last" array element is also an issue. A far better approach is to instead actually add "new items" to the "beginning" of the array, instead of the "end". This is actually what the $position or possibly even the $sort modifiers to $push do for you, by changing where items get added or the sorted order of items respectively.



Changing the Array to "newest first"



This takes a little work by changing the way you store things, but the benefits are greatly improved speed of such queries like you want without needing an evaluated $expr argument.



The core concepts are to "pre-pend" new array items with syntax like:



$this->collection->updateOne(
$query,
[ '$push' => [ 'allotment_details' => [ '$each' => $allotments, '$position' => 0 ] ] ]
)


Where $alloments must be an array as required by $each and $position is used to 0 in order to add the new array item "first".



Alternately if you actually have something like created_date as a property within each of the objects in the array, then you "could" use something like $sort as a modifier instead.



$this->collection->updateOne(
$query,
[ '$push' => [
'allotment_details' => [ '$each' => $allotments, '$sort' => [ 'created_date' => -1 ] ]
]]
)


It really depends on whether your "query" and other access requirements rely on "last added" or "latest date", and then also typically if you intend to possibly alter such a created_date or other "sort" property in a way which would effect the order of the array elements when "sorted".



The reason you do this is then matching the "latest" ( which is now the "first" ) item in the array simply becomes:



$this->collection->find([
'allotment_details.0.room_id': $this->RoomId
])


MongoDB allows the "first" array index to be specified with "Dot Notation", using the 0 index. What you cannot do is specify a "negative" index i.e:



$this->collection->find([
'allotment_details.-1.room_id': $this->RoomId # not allowed :(
])


That's the reason why you do the things shown above on "update" in order to "re-order" your array to the workable form.



Concatenation is Bad



The other main issue is the concatenation of strings. As already mentioned this creates unnecessary overhead just in order to do the matching you want. It's also "unnecessary" since you can complete avoid this using $or with the conditions on each of the fields as they exist already within the actual document:



 $this->collection->find([
'$or' => [
[ 'first_name' => new MongoDBBSONRegex($arg, 'i') ],
[ 'last_name' => new MongoDBBSONRegex($arg, 'i') ],
[ 'middle_name' => new MongoDBBSONRegex($arg, 'i') ],
[ 'registration_temp_perm_no' => $arg ]
],
'schoolId' => new MongoDBBSONObjectID($this->SchoolId),
'allotment_details.0.room_id': $this->RoomId
])


And of course whatever the "full" query conditions actually need to be, but you should be getting the basic idea.



Also if you are not actually looking for "partial words", then a "text search" defined over the fields with the "names". After creating the index that would be:



 $this->collection->find([
'$text' => [ '$search' => $arg ],
'schoolId' => new MongoDBBSONObjectID($this->SchoolId),
'allotment_details.0.room_id': $this->RoomId
])




Overall I would really recommend looking closely at all the other options rather than making one small change to your existing code. With a little careful re-structuring of how you store things and indeed "index" things, you get huge performance benefits that your extensive $concat "brute force" approach simply cannot deliver.




N.B Modern PHP Releases generally support as a much more brief representation of array(). It's a lot cleaner and far easier to read. So please use it.








share|improve this answer














share|improve this answer



share|improve this answer








edited Nov 14 at 2:53

























answered Nov 13 at 10:55









Neil Lunn

96.3k22169179




96.3k22169179












  • Thanks for the effort u have put to answer my question
    – Nisa Nisa
    Nov 13 at 12:25


















  • Thanks for the effort u have put to answer my question
    – Nisa Nisa
    Nov 13 at 12:25
















Thanks for the effort u have put to answer my question
– Nisa Nisa
Nov 13 at 12:25




Thanks for the effort u have put to answer my question
– Nisa Nisa
Nov 13 at 12:25


















draft saved

draft discarded




















































Thanks for contributing an answer to Stack Overflow!


  • Please be sure to answer the question. Provide details and share your research!

But avoid



  • Asking for help, clarification, or responding to other answers.

  • Making statements based on opinion; back them up with references or personal experience.


To learn more, see our tips on writing great answers.





Some of your past answers have not been well-received, and you're in danger of being blocked from answering.


Please pay close attention to the following guidance:


  • Please be sure to answer the question. Provide details and share your research!

But avoid



  • Asking for help, clarification, or responding to other answers.

  • Making statements based on opinion; back them up with references or personal experience.


To learn more, see our tips on writing great answers.




draft saved


draft discarded














StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53278310%2fexpr-arrayelementat-not-working-in-aggregation-for-embedded-document%23new-answer', 'question_page');
}
);

Post as a guest















Required, but never shown





















































Required, but never shown














Required, but never shown












Required, but never shown







Required, but never shown

































Required, but never shown














Required, but never shown












Required, but never shown







Required, but never shown







Popular posts from this blog

To store a contact into the json file from server.js file using a class in NodeJS

Redirect URL with Chrome Remote Debugging Android Devices

Dieringhausen