Scala Remove Duplicates From List: Distinct Example

Use the distinct function, along with map, to remove duplicate elements from lists.

Remove duplicates. A list contains strings: chair, table, table. It has a duplicate string. With Scala we have many ways to remove duplicates from our list.

With distinct, a method on the List type, we eliminate duplicates and retain a list's order. Other approaches are possible. We can convert the list to a set: this also dedupes a list.

An example. Let us begin with this example. We create a constant, immutable List of furniture strings. The list has two instances of the string "table."
Distinct: We invoke the distinct function on the furniture list. A new list, with duplicates removed, is returned.
Scala program that removes duplicates from list // Create a list of strings. val furniture = List("chair", "bed", "table", "table", "couch") // Get distinct strings. // ... This removes duplicates but retains order. val result = furniture.distinct // Print results. println(furniture) println(result) Output List(chair, bed, table, table, couch) List(chair, bed, table, couch)

ToSet, toList. Here we use toSet and toList to strip duplicate Ints. We create a list that has six Ints, and two duplicate Ints. We then remove those duplicates.
ToSet: This converts the list to a set. Duplicates are removed because a set cannot store duplicates.
ToList: We convert the set back into a list. Please note that ordering may be changed by the set.
Scala program that uses toSet, toList // Create a list of Ints. val ids = List(10, 10, 1, 2, 3, 3) println(ids) // Convert list to set. // ... Duplicate elements are removed at this step. val set = ids.toSet println(set) // Convert set to list. val ids2 = set.toList println(ids2) Output List(10, 10, 1, 2, 3, 3) Set(10, 1, 2, 3) List(10, 1, 2, 3)

Map, distinct. With the map method we can transform all elements into a standard form. This may result in duplicates. With distinct we can remove the dupes.
Here: We use a lambda expression to map all strings to uppercase forms. A more complex method could be applied.
Scala program that uses map and distinct val codes = List("abC", "Abc", "ABC", "xyz", "XyZ") println(codes) // Convert all strings to uppercase. // ... Then get distinct strings. val result = codes.map(_.toUpperCase()).distinct println(result) Output List(abC, Abc, ABC, xyz, XyZ) List(ABC, XYZ)

Sets, maps. Some collections enforce unique elements. These never need to have duplicates removed—duplicates never occur. Sets and maps cannot store duplicates.SetMap

A summary. Removing duplicates from a list is an essential task in programming. It helps us understand basic manipulations of lists. And it is useful in real programs.

© 2007-2020 Sam Allen. Every person is special and unique. Send bug reports to info@dotnetperls.com.