Home>

a.txt reads as follows:

123

456

789

123

123

789

Get duplicate rows,And duplicate lines are kept only once,After processing, the content of b.txt is as follows:

123

789

method 1:

@echo off
rem disadvantage 1:cannot handle extremely large files
rem cons 2:Need to mark variables with characters that don't appear in the file (underscores are used in this example)
setlocal
for/f "delims =" %%i in (a.txt) do (
  set/a _ %%i +=1
)
(for/f "tokens=1-2 delims=_ =" %%i in ("set _") do (
  if %%j gtr 1 (
    echo, %%i
  )
))>B.txt
endlocal

Method 2:

@echo off
setlocal enabledelayedexpansion
set "priline ="
set "dupnum=1"
(for/f "delims =" %%i in ("sort a.txt") do (
  if "! priline!" equ "%%i" (
    set/a dupnum +=1
  ) else (
    if! dupnum! gtr 1 (
      echo,! priline!
    )
    set dupnum=1
  )
  set "priline=%%i"
))>B.txt
if! dupnum! gtr 1 (
  >>b.txt echo,! Priline!
)

Method 3:

gawk "{a [$0] ++} end {for (i in a) if (a [i]>1) print i}" a.txt>b.txt

Method 4:(more concise)

gawk "a [$0] ++" a.txt>b.txt

Method 5:

@echo off
for/f "tokens=*" %%i in (a.txt) do set #%%i=%%i
(for/f "tokens=2 delims ==" %%i in ("set #") do echo %%i)>b.txt
  • Previous Four recognitions that userAgent userAgent can achieve
  • Next Implement ios like function based on UIControl control