aggregate allows a stage type only once

[oallouch]

We often need to use a stage type (specially project and match) several times in the same pipeline, but the JS-SDK's ParseQuery (the Node part of course) prevents it in its aggregate method:

if (Array.isArray(pipeline)) {
   pipeline.forEach((stage) => {
     for (let op in stage) {
       stages[op] = stage[op];
     }
   });
 }

Note that, unlike Map, an Object doesn't guaranty order.

For now, I use this workaround:

const Config = require('../../node_modules/parse-server/lib/Config');

async function aggregate(className, pipeline) {
	//---- $ adding in stage name ----//
	pipeline = pipeline.map(stage => {
		// stage is an object with a single key
		const stageName = Object.keys(stage)[0];
		return { ['$' + stageName]: stage[stageName] };
	})
	//---- direct execution ----//
	const config = Config.get(Parse.applicationId);
	return await config.database.adapter.aggregate('_User', { fields: {} }, pipeline, 'SECONDARY_PREFERRED');
}

Hope that helps.

Olivier, from Paris

[dplewis]

We often need to use a stage type (specially project and match) several times in the same pipeline

I can't think of a reason to use the same stage more than once, but if it is supported we should have it here.

Do you have a pipeline you use that shows this issue? A test case would also help.

Note that, unlike Map, an Object doesn't guaranty order.

During my tests the objects were in order all of the time. I do agree that if we pass around an Array instead order will be guaranteed.

[oallouch]

For instance, you often add a $match (or a project) after a $lookup.
And you often add a $project before a $lookup, to prepare the joined fields.

[dplewis]

I get why ordering is essential. My question is about same stages being used more than once. I can submit a PR if you have an example I can test with.

[oallouch]

Ok. I didn't test this code, but it's the global idea:

const loadTestData = () => {
	const data1 = {score: 10, name: 'foo', sender: {group: 'A'}, views: 900, size: ['S', 'M']};
	const data2 = {score: 10, name: 'foo', sender: {group: 'A'}, views: 800, size: ['M', 'L']};
	const data3 = {score: 10, name: 'bar', sender: {group: 'B'}, views: 700, size: ['S']};
	const data4 = {score: 20, name: 'dpl', sender: {group: 'B'}, views: 700, size: ['S']};
	const obj1 = new TestObject(data1);
	const obj2 = new TestObject(data2);
	const obj3 = new TestObject(data3);
	const obj4 = new TestObject(data4);

	const user1 = new Parse.User({name: 'John', obj: obj1});
	const user2 = new Parse.User({name: 'Michael', obj: obj1});
	const user3 = new Parse.User({name: 'Franck', obj: obj2});

	return Parse.Object.saveAll([obj1, obj2, obj3, obj4, user1, user2, user3]);
}

Say, I want all the datas from group A which are linked to, at least 1 user.

const pipeline = [
	{
		match: { 'sender.group': 'A' }
	}, {
		project: { objectId: 1, objPointer: { '$concat' : [ 'TestObject$', '$_id' ] } }
	}, {
		lookup: { from : '_User', localField : 'objPointer', 'foreignField' : '_p_obj', 'as' : 'users' }
	}, {
		match: { objectId: 1, users: { $ne: [] } }
	}
];

const response = await new Parse.Query('TestObject').aggregate(pipeline, { userMasterKey: true });

$lookup adds more data on which we can apply new $match or $project stages, for instance.

I hope the code makes sense,
Olivier

[milad145]

i have this problem too?
and i think its a big problem, so If this problem will be resolved, I will be grateful
thanks
with regards

[oallouch]

For now, you can use the little workaround I've put above.
Remember that it's only from the server to the server.