Menghapus semua duplikat

Saya mencoba menghapus semua duplikat tetapi hanya menyimpan satu catatan (id lebih pendek). Permintaan berikut menghapus duplikat tetapi mengambil banyak iterasi untuk menghapus semua salinan dan menyimpan yang asli.

DELETE FROM emailTable WHERE id IN (
 SELECT * FROM (
    SELECT id FROM emailTable GROUP BY email HAVING ( COUNT(email) > 1 )
 ) AS q
)

MySQL-nya.

Edit # 1 DDL

CREATE TABLE `emailTable` (
 `id` mediumint(9) NOT NULL auto_increment,
 `email` varchar(200) NOT NULL default '',
 PRIMARY KEY  (`id`)
) ENGINE=MyISAM AUTO_INCREMENT=298872 DEFAULT CHARSET=latin1

Sunting # 2 Ini berfungsi seperti pesona oleh @Dtest

DELETE FROM emailTable WHERE NOT EXISTS (
 SELECT * FROM (
    SELECT MIN(id) minID FROM emailTable    
    GROUP BY email HAVING COUNT(*) > 0
  ) AS q
  WHERE minID=id
)

Coba ini:

DELETE FROM emailTable WHERE NOT EXISTS (
 SELECT * FROM (
    SELECT MIN(id) minID FROM emailTable    
    GROUP BY email HAVING COUNT(*) > 0
  ) AS q
  WHERE minID=id
)

Di atas berfungsi untuk pengujian saya terhadap 50 email (5 email berbeda digandakan 10 kali).

Anda mungkin perlu menambahkan indeks pada kolom 'email':

ALTER TABLE emailTable ADD INDEX ind_email (email);

Mungkin agak lambat untuk 250.000 baris. Itu lambat bagi saya di atas meja yang memiliki 1,5 juta baris (diindeks dengan benar), itulah bagaimana saya datang dengan strategi ini:

/* CREATE MEMORY TABLE TO HOUSE IDs of the MIN */
CREATE TABLE email_min (minID INT, PRIMARY KEY(minID)) ENGINE=Memory;

/* INSERT THE MINIMUM IDs */
INSERT INTO email_min SELECT id FROM email
    GROUP BY email HAVING MIN(id);

/* MAKE SURE YOU HAVE RIGHT INFO */
SELECT * FROM email 
 WHERE NOT EXISTS (SELECT * FROM email_min WHERE minID=id)

/* DELETE FROM EMAIL */
DELETE FROM email 
 WHERE NOT EXISTS (SELECT * FROM email_min WHERE minID=id)

/* IF ALL IS WELL, DROP MEMORY TABLE */
DROP TABLE email_min;

Manfaat dari tabel memori adalah ada indeks yang digunakan (kunci utama pada minID) yang mempercepat proses lebih dari tabel sementara yang normal.

Berikut adalah proses penghapusan yang lebih ramping:

CREATE TABLE emailUnique LIKE emailTable;
ALTER TABLE emailUnique ADD UNIQUE INDEX (email);
INSERT IGNORE INTO emailUnique SELECT * FROM emailTable;
SELECT * FROM emailUnique;
ALTER TABLE emailTable  RENAME emailTable_old;
ALTER TABLE emailUnique RENAME emailTable;
DROP TABLE emailTable_old;

Berikut ini beberapa contoh data:

use test
DROP TABLE IF EXISTS emailTable;
CREATE TABLE `emailTable` (
 `id` mediumint(9) NOT NULL auto_increment,
 `email` varchar(200) NOT NULL default '',
 PRIMARY KEY  (`id`)
) ENGINE=MyISAM;
INSERT INTO emailTable (email) VALUES
('redwards@gmail.com'),
('redwards@gmail.com'),
('redwards@gmail.com'),
('redwards@gmail.com'),
('rolandoedwards@gmail.com'),
('rolandoedwards@gmail.com'),
('rolandoedwards@gmail.com'),
('red@gmail.com'),
('red@gmail.com'),
('red@gmail.com'),
('rolandoedwards@gmail.com'),
('rolandoedwards@gmail.com'),
('rolandoedwards@comcast.net'),
('rolandoedwards@comcast.net'),
('rolandoedwards@comcast.net');
SELECT * FROM emailTable;

Saya berlari mereka. Inilah hasilnya:

mysql> use test
Database changed
mysql> DROP TABLE IF EXISTS emailTable;
Query OK, 0 rows affected (0.01 sec)

mysql> CREATE TABLE `emailTable` (
    ->  `id` mediumint(9) NOT NULL auto_increment,
    ->  `email` varchar(200) NOT NULL default '',
    ->  PRIMARY KEY  (`id`)
    -> ) ENGINE=MyISAM;
Query OK, 0 rows affected (0.05 sec)

mysql> INSERT INTO emailTable (email) VALUES
    -> ('redwards@gmail.com'),
    -> ('redwards@gmail.com'),
    -> ('redwards@gmail.com'),
    -> ('redwards@gmail.com'),
    -> ('rolandoedwards@gmail.com'),
('rolandoedwards@comcast.net');
SELECT * FROM emailTable;
    -> ('rolandoedwards@gmail.com'),
    -> ('rolandoedwards@gmail.com'),
    -> ('red@gmail.com'),
    -> ('red@gmail.com'),
    -> ('red@gmail.com'),
    -> ('rolandoedwards@gmail.com'),
    -> ('rolandoedwards@gmail.com'),
    -> ('rolandoedwards@comcast.net'),
    -> ('rolandoedwards@comcast.net'),
    -> ('rolandoedwards@comcast.net');
Query OK, 15 rows affected (0.00 sec)
Records: 15  Duplicates: 0  Warnings: 0

mysql> SELECT * FROM emailTable;
+----+----------------------------+
| id | email                      |
+----+----------------------------+
|  1 | redwards@gmail.com         |
|  2 | redwards@gmail.com         |
|  3 | redwards@gmail.com         |
|  4 | redwards@gmail.com         |
|  5 | rolandoedwards@gmail.com   |
|  6 | rolandoedwards@gmail.com   |
|  7 | rolandoedwards@gmail.com   |
|  8 | red@gmail.com              |
|  9 | red@gmail.com              |
| 10 | red@gmail.com              |
| 11 | rolandoedwards@gmail.com   |
| 12 | rolandoedwards@gmail.com   |
| 13 | rolandoedwards@comcast.net |
| 14 | rolandoedwards@comcast.net |
| 15 | rolandoedwards@comcast.net |
+----+----------------------------+
15 rows in set (0.00 sec)

mysql> CREATE TABLE emailUnique LIKE emailTable;
Query OK, 0 rows affected (0.04 sec)

mysql> ALTER TABLE emailUnique ADD UNIQUE INDEX (email);
Query OK, 0 rows affected (0.06 sec)
Records: 0  Duplicates: 0  Warnings: 0

mysql> INSERT IGNORE INTO emailUnique SELECT * FROM emailTable;
Query OK, 4 rows affected (0.01 sec)
Records: 15  Duplicates: 11  Warnings: 0

mysql> SELECT * FROM emailUnique;
+----+----------------------------+
| id | email                      |
+----+----------------------------+
|  1 | redwards@gmail.com         |
|  5 | rolandoedwards@gmail.com   |
|  8 | red@gmail.com              |
| 13 | rolandoedwards@comcast.net |
+----+----------------------------+
4 rows in set (0.00 sec)

mysql> ALTER TABLE emailTable  RENAME emailTable_old;
Query OK, 0 rows affected (0.03 sec)

mysql> ALTER TABLE emailUnique RENAME emailTable;
Query OK, 0 rows affected (0.00 sec)

mysql> DROP TABLE emailTable_old;
Query OK, 0 rows affected (0.00 sec)

mysql>

Seperti yang ditunjukkan, emailTable akan berisi kemunculan pertama setiap alamat email dan id asli yang sesuai. Untuk contoh ini:

• ID 1-4 memiliki redwards@gmail.com, tetapi hanya 1 yang dipertahankan.
• ID 5-7,11,12 memiliki rolandoedwards@gmail.com, tetapi hanya 5 yang dipertahankan.
• ID 8-10 memiliki red@gmail.com, tetapi hanya 8 yang dipertahankan.
• ID 13-15 memiliki rolandoedwards@comcast.net, tetapi hanya 13 yang dipertahankan.

CAVEAT: Saya menjawab pertanyaan yang mirip dengan ini tentang penghapusan tabel melalui pendekatan tabel temp .

Cobalah !!!

Inilah solusi Itzik cepat nyata. Ini akan bekerja di SQL 2005 dan lebih tinggi.

WITH Dups AS
(
  SELECT *,
    ROW_NUMBER()
      OVER(PARTITION BY email ORDER BY id) AS rn
  FROM dbo.emailTable
)
DELETE FROM Dups
WHERE rn > 1;