Remove duplicates from a list in python
By: Python Documentation Team
If you don’t mind reordering the list, sort it and then scan from the end of the list, deleting duplicates as you go:
if mylist:
mylist.sort()
last = mylist[-1]
for i in range(len(mylist)-2, -1, -1):
if last == mylist[i]:
del mylist[i]
else:
last = mylist[i]
If all elements of the list may be used as dictionary keys (i.e. they are all hashable) this is often faster
d = {}
for x in mylist:
d[x] = 1
mylist = list(d.keys())
In Python 2.5 and later, the following is possible instead:
mylist = list(set(mylist))
This converts the list into a set, thereby removing duplicates, and then back into a list.
You can also see the Python Cookbook for a long discussion of many ways to do this:
http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/52560
Archived Comments
Comment on this tutorial
- Data Science
- Android
- AJAX
- ASP.net
- C
- C++
- C#
- Cocoa
- Cloud Computing
- HTML5
- Java
- Javascript
- JSF
- JSP
- J2ME
- Java Beans
- EJB
- JDBC
- Linux
- Mac OS X
- iPhone
- MySQL
- Office 365
- Perl
- PHP
- Python
- Ruby
- VB.net
- Hibernate
- Struts
- SAP
- Trends
- Tech Reviews
- WebServices
- XML
- Certification
- Interview
categories
Related Tutorials
Python program to get location meta data from an image
Retrieve Twitter posts and comments using Python
How to install Jupyter in Ubuntu and make it accessible through Apache Reverse Proxy
Python Basics - Setting up your Python Development Environment
Schwartzian Transform in python
Multidimensional list (array) in python
Perl's chomp() equivalent for removing trailing newlines from strings in python