A performant way to work with obtusely large files in PowerShell #Quickie #PowerShell
I have a pattern that makes working with large text files one-line-at-a-time inpowershelll much more performant by reading in a block of lines and parsing them in memory instead of reading one line at a time.
I've forgotten and re-invented this enough times that I'm writing it here so future me can find it.
get-content $filename -Encoding Unicode -ReadCount 10000 | foreach-object {
"Working line $i"
$i+=10000
foreach ($line in $_) {
# do something
}
}
Comments