hive表数据去重

根据user name查重 SELECT test_user_name, count(*) c from default.test GROUP BY test_user_name HAVING c> 1; 根据id查重 SELECT id ,count(*) c FROM default.test GROUP BY id HAVING c >1; 去重：注意min用法，取相同数据的最小id，去重的重点 min:使用group by 分组后取分组内最小id，以便去掉其它重复数据 INSERT OVERWRITE table default.test partition(test_data_source) select * from default.test where id in ( select min(id) as id from default.test group by test_user_name) ;

发布者：全栈程序员栈长，转载请注明出处：https://javaforall.cn/100394.html原文链接：