Merge ans deduplicate domains in a blocklist
Use WGET to download lists, then combine them into a single large file, and finally create a new file with no duplicates by using “awk '!visited[$0]++'”
wget URL1 URL2 URL3 cat *.txt > all.txt (This overwrites all.txt)
awk '!visited[$0]++' all.txt > no_duplicates.txt
Or use chatgptlol
No Comments