how to remove duplicates from collection

[AmitLoh-Powerap]

I have a collection created with duplicate records. Distinct give me single column record which I do not want.

I just want to remove duplicate rows from collection based on one column but it should return collection again(not one column)

how to remove duplicates from collection?

[v-micsh-msft]

Hi @AmitLoh-Powerap,

Thanks for the question.

Currently I can't find a way available to remove duplicate records from collection in PowerApps.

The only function that would work with the Duplicate records is Distinct function, and it will only return single columns.

We could archive the duplicate records by using GrouPBy function:

GroupBy and Ungroup functions in PowerApps

Please check to see if the groupby function could be a workaround.

In addition, please also submit an idea regarding delete the duplicate records.

Regards

[Anonymous · Accepted solution]

I found a way to remove duplicates. It is not efficient, but it works!

1- create a column in the collection that concatenates all values that can define the tuple as unique.

2 - create a "temporary" collection with the distinct values from this new column. 

ClearCollect(tempCol,Distinct(originalCollection, NEWCOLUMN));

3- execute a loop in this tempCol and filtering and getting the first values from the originalCollection, and storing them in a second "temporary" collection.

ForAll(tempCol,Collect(tempCol2,First(Filter(originalCollection , NEWCOLUMN=Result))));

4- Replace the originalCollection by the second "temporary" collection.

ClearCollect(projectLocation,tempCol2)

5- its done! you have a new collection with all duplicated values removed. 

OBS: Make sure you concatenated the correct columns in order to make the tuple unique.

[james_hathaway]

I found a pretty good way that uses Group and UnGroup:

col_Original = Original Collection that contains Duplicates

col_Distinct = Collection in the same shape as the original (Same Columns), but without Duplicates.

UnGroup(
   DropColumns(
      AddColumns(
         GroupBy(col_Original,"ColumnToDistinctOn","Rows"),
         "FirstRow",
         Table(First(Rows))
     ),
     "Rows"
   ),
   "FirstRow"
)

Adding the Column "FirstRow" as a TABLE and Not a Record is key to making the Ungroup work properly.

Dropping the Column "Rows" is not strictly necessary, and might actually be useful if you need it, but I included it to make sure the end Collection has the same columns as the original.

As other have said on this thread, if you don't have a "ColumnToDistinctOn", just use AddColumns to create one.

Wrap this above formula in a "ClearCollect(col_Distinct, [....] )" function to use the collection elsewhere in your app.

Hope this helps!

James.

[volly]

I was able to use the help of the posts above to solve my problem so I thought I would share my case and my outcome.

I had a similar problem where a user can select multiple building descriptions in a combo box and I want to display the associated building classifications. The combo box is linked to a sharepoint table. This is some basic data from the table

Title	BuildingDescription
Class 1a	Private Residence
Class 1a	Duplex Residence
Class 1a	Town House
Class 1a	Terrace House
Class 1b	Boarding House

 I have a combo box named: pd-BuildingDescriptions. when the user selects these items below,

Terrace House
Town House
Duplex Residence
Private Residence

All of these types are of Class 1a. 

I have used this formula to get the associated classes

TrimEnds(Concat('pd-BuildingDescriptions'.SelectedItems,Title &" "))

I am using three spaces to act as a delimiter so I can eaisly trim the last delimeter with TrimEnds.

This results in a string "Class 1a   Class 1a   Class 1a   Class 1a"

The formula below provides only unique results of the building class

Set(varString,
Concat(
     GroupBy(
          Split(
          TrimEnds(Concat('pd-BuildingDescriptions'.SelectedItems,TrimEnds(Title) &"  "))
          ,"  "),
     "Result","Results"),
Result & " ")
)

I built it by loading the variables below into a single line formula. 

Set(varRepeatingString,
    TrimEnds(
        Concat('pd-BuildingDescriptions'.SelectedItems,Title & " ")
    )
); //End Set varRepeatingString

Set(varDelimiter,"   "); // three spaces for my case

Set(
    varString,
    Concat(
        GroupBy(
            Split(
                varRepeatingString,
                varDelimiter
            )//close split
            ,
            "Result", //Apply grouping to this column name. 
            "GroupedColumnName" //Name of new column. Redundant
        )//close groupby
        ,
        Result & varDelimiter)//close concat
)//close set varString

The end result is varString = "Class 1a"

Thanks everyone for your contributions. 

[mdevaney · Accepted solution]

@AmitLoh-Powerap 

You can remove the duplicate rows with this one-line of code.

//Duplicate rows removal code
ClearCollect(colSolution, ForAll(Distinct(yourCollectionName, ThisRecord), Result));

For a full example of how it works check out my article Remove Duplicate Rows From A Power Apps Collection.

https://www.matthewdevaney.com/remove-duplicate-rows-from-a-power-apps-collection/

---
Please click "Accept as Solution" if my post answered your question so that others may find it more quickly. If you found this post helpful consider giving it a "Thumbs Up."

[LoongWang]

Error: 'Result' isn't recognized

[james_hathaway]

Ah, Yes,

I think in the time since these posts, they've changed the Split function - I think it creates an array of "Value"s now, not "Result"s...

[rkirk2k]

NOTE: Microsoft made a change to the "ForAll" function in Power FX. That code will not work, and there's no indication in the error as to why. They replaced "Result" with "Value" (I GUESS to be consistent with all the other function syntax).

But THIS works now...

//Duplicate rows removal code
ClearCollect(colSolution, ForAll(Distinct(yourCollectionName, ThisRecord), Value));

[SK_Daniel]

The corrected code:

//Duplicate rows removal code
ClearCollect(colSolution, ForAll(Distinct(yourCollectionName, ThisRecord), Value));

@mdevaney  Please update the website page with this correction. Nice website btw!