For example, there is now a Person class:

public class Person {
        private Long id;
        private String name;

        public Person(Long id, String name) {
            this.id = id;
            this.name = name;
        }

        public Long getId() {
            return id;
        }

        public void setId(Long id) {
            this.id = id;
        }

        public String getName() {
            return name;
        }

        public void setName(String name) {
            this.name = name;
        }

        @Override
        public String toString() {
            return "Person{" +
                    "id=" + id +
                    ", name='" + name + '\'' +
                    '}';
        }

    }

Override the equals() method and hashCode() method of the Person object:

@Override
    public boolean equals(Object o) {
        if (this == o) return true;
        if (o == null || getClass() != o.getClass()) return false;
        Person person = (Person) o;
        if (!id.equals(person.id)) return false;
        return name.equals(person.name);
    }


    @Override
    public int hashCode() {
        int result = id.hashCode();
        result = 31 * result + name.hashCode();
        return result;
    }

1. De duplication according to the attribute of the object

(1) De duplication by single attribute

Next, we need to redo according to the id of the Person object. What should we do?

Write a method:

public static List<Person> removeDupliById(List<Person> persons) {
        Set<Person> personSet = new TreeSet<>((o1, o2) -> o1.getId().compareTo(o2.getId()));
        personSet.addAll(persons);
        return new ArrayList<>(personSet);
    }

Through the Comparator comparator, compare the object properties, and return 0 if they are the same, so as to achieve the purpose of filtering.

Let's look at the cool way of writing Java 8:

public static void main(String[] args) {
        // De duplication according to id
        List<Person> unique = persons.stream().collect(collectingAndThen(
                toCollection(() -> new TreeSet<>(comparingLong(Person::getId))), ArrayList::new)
        );
    }

public static void main(String[] args) {
        // De duplication according to name
        List<Person> unique = persons.stream().collect(Collectors.collectingAndThen(
                Collectors.toCollection(() -> new TreeSet<>(Comparator.comparing(Person::getName))), ArrayList::new));
        unique.forEach(p -> System.out.println(p));
    }

(2) De duplication by multiple attributes

public static void main(String[] args) {
        List<Person> list = new ArrayList<>();
        list.add(new Person(1L, "java"));
        list.add(new Person(1L, "C"));
        list.add(new Person(2L, "C++"));
        list.add(new Person(1L, "java"));
        list.add(new Person(1L, "java"));
        Set<Person> set = new TreeSet<>(Comparator.comparing(Person::getId).thenComparing(Person::getName));
        set.addAll(list);
        System.out.println(set);
    }

2. De duplication of objects in the list

@Override
    public boolean equals(Object o) {
        if (this == o) return true;
        if (o == null || getClass() != o.getClass()) return false;
        Person person = (Person) o;
        if (!id.equals(person.id)) return false;
        return name.equals(person.name);
    }


    @Override
    public int hashCode() {
        int result = id.hashCode();
        result = 31 * result + name.hashCode();
        return result;
    }

The following object de duplication code:

public static void main(String[] args) {
        Person p1 = new Person(1l, "jack");
        Person p2 = new Person(3l, "jack chou");
        Person p3 = new Person(2l, "tom");
        Person p4 = new Person(4l, "hanson");
        Person p5 = new Person(5l, "Tape worm");

        List<Person> persons = Arrays.asList(p1, p2, p3, p4, p5, p5, p1, p2, p2);

        List<Person> personList = new ArrayList<>();

        // duplicate removal
        persons.stream().forEach(p -> {
            if (!personList.contains(p)) {
                personList.add(p);
            }
        });

        System.out.println(personList);

    }

The underlying implementation of the contains() method of List uses the equals method of the object to compare. In fact, it's good to rewrite equals(), but it's better to rewrite hashCode if equals is rewritten.

public static void main(String[] args) {
        // De duplication according to id
        List<Person> unique = persons.stream().collect(collectingAndThen(
                toCollection(() -> new TreeSet<>(comparingLong(Person::getId))), ArrayList::new)
        );
    }

public static void main(String[] args) {
        // De duplication according to name
        List<Person> unique = persons.stream().collect(Collectors.collectingAndThen(
                Collectors.toCollection(() -> new TreeSet<>(Comparator.comparing(Person::getName))), ArrayList::new));
        unique.forEach(p -> System.out.println(p));
    }

 3. De duplication of simple elements in list (String, etc.)

public List<String> removeStringListDupli(List<String> stringList) {
    Set<String> set = new LinkedHashSet<>();
    set.addAll(stringList);
    stringList.clear();
    stringList.addAll(set);
    return stringList;
}

Or use Java 8:

List<String> unique = list.stream().distinct().collect(Collectors.toList());

The aggregation operation collect of the stream API of the people list allows us to focus only on the results, and the collectingAndThen in the collect method belongs to Java util. stream. The explanation of collector, collectingAndThen operation is: first perform the previous operation, then perform the second operation and output the result.

The first step is collectors toCollection(() -> new TreeSet<>(Comparator.comparing(Person::getName)))；

The second step is to output it as a new ArrayList;

In the first step, the Collectors interface is used again. This time, the toCollection method is used, which is to convert the functions or parameters in the method into a collection. Here, we use comparator Comparing (person:: getname) is transformed into a collection, which is a TreeSet. That is, orderly. Because we need to get rid of duplicate values, this set can do it, and we need to keep the transformed collection in order, so we use a TreeSet.

Comparator.comparing(Person::getName) here, Java is used again util. Comparator interface, which is quite commonly used. His comparing method is used to compare whether the values of parameters are the same. The lambda expression, a new feature of java8, is actually the same as It doesn't make much difference. For example, the most commonly used system out. Println () can be expressed as system Out:: println can achieve the same effect.