Evolving Code Monkey

Since last month, I’ve made a small change to the way I rename the files produced by my scanner.

I wrote the following short function:

function nuname ($n) {
   [void]($n -match "Page(\d+)_(\d+)")
   return "Page" + [string]([int]$matches[1] + [int]$matches[2]) + ".png"
}

And now, instead of naming files “Page ###”, I simply export them from the HP scanner dialog using the number of the page before the first page I scanned. The HP scanner software adds a sequence number starting with 1, so my function adds the number I supplied and the sequence number from the software, then fills that in as the page number in the resulting file name.

For example, if I start scanning a number of pages starting with page 180 from the book, I save the bunch as
179.png. The HP scanner software saves that page as 179_1.png, and my function turns that into Page180.png
.

Of course, it would probably make more sense to name the bunch with the actual page number of the first page scanned, in which case the function would have to be changed slightly:

function nuname ($n) {
   [void]($n -match "Page(\d+)_(\d+)")
   return "Page" + [string]([int]$matches[1] + [int]$matches[2] - 1) + ".png"
}

Then, when I scanned a bunch starting with Page 180, I would save the bunch as 180.png, the first one would be saved (by the scanner software) as 180_1.png, and my function would change it to Page180.png. Much more intuitive.

I’ve been scanning pages from a paperback book (by the way, did you know that Kinko’s / Fedex stores often have a paper cutter capable of cutting the spine off a paperback book, making it very easy to scan those pages?).

My new HP scanner will save files in a sequence – but each time I scan, I have to figure out to save the files so that the file name matches the page number.

For example, I started by scanning pages 18 through 29, and I saved the first file as “Page018.png”, hoping that the subsequent pages would be saved as “Page019.png”, “Page020.png”…”Page029.png”

However, what I got was “Page018_0001.png”, “Page018_0002.png”…”Page018_0012.png”

I used PowerShell to rename the files –

> $pages = Get-ChildItem c:\scans\1
> $pages |
  Rename-Item -NewName {$_.Name -replace "\d\d_\d{4}",
           [string](17 + [int]($_.Name).Substring(9,3)) }

For the next batch, I scanned pages 32 through 54, and saved the first one as “Page.png” – the pages were saved as “Page_0001.png”, “Page_0002.png”…”Page_0023.png”

This time, I had to use the following commands –

> $pages = Get-ChildItem c:\scans\1
> $pages | 
   Rename-Item -NewName {$_.Name -replace "_\d{4}", 
            ('00' + [string](31 + [int]($_.Name).Substring(7,2))) }

I do wish I could save these with the right sequence start and not have to rename them.

Other than this quirk, I confess I’m enjoying this HP device. It’s got its own wireless NIC, so I can scan directly to my home file server, and I don’t have to mess with USB on my laptop.

Searching a big text file in PowerShell

Where I work, we use simple text files as menus for our web applications. These menus may reference hundreds of jobs per month, and span up to 84 months (7 years) in some cases.

Looking up lines in these files can be very time-consuming. Since I’m writing scripts that have to fit around our processing automation, I usually can’t sort the files (or alter them in any other way, in fact) to make it possible to do some sort of binary search on them.

Doing a linear search might be workable if the lines I wanted were near the beginning of the files, but I usually want the information near the end (since new lines are appended to the end of the file). Also, the file server where the local copy of these files reside has historically been very dodgy, and communicating with it over our LAN has caused problems with normal processing, so I wouldn’t want to add to that load by doing Get-Content in my PowerShell script and passing the data line-by-line to the search logic.

Mind you, that server has been replaced recently, and the new one works much better (I think they upgraded from Windows Server 2003 to Server 2012, skipping right over Server 2008 – which gives you an idea how long we suffered with the previous incarnation).

Still, it’s better not to flog the network with more traffic than necessary, and in any case old habits die hard. So how do we make searching a plain text file across the LAN more efficient?

Well, for starters, I copy the entire file into a variable on my PC, so the whole thing is stored in RAM. We have ordinary desktop PCs, nothing high-end, but just about any PC you’ll find in a business environment these days will have at least 4 GB of RAM. A menu file with hundreds of thousands of lines will have an actual size on disk of just a few MB – so grab the entire file.

$big_menu = Get-Content \\Serv1\jobs\cust37\menufiles\MEMBERS.TXT

Since $big_menu is an array of strings, it’s not out of the question to do a brute-force linear search for one or two lines. If the lines you want are most likely to be at the end of the file, it’s possible to step backwards through an array of strings in a way that isn’t feasible when you’re reading the file line-by-line from disk.

Or if you know that, say, 117 jobs were processed today, and you just want the last 117 lines of the menu, you can create a cut-down version of the menu using the Select-Object cmdlet:

$todays_menu = $big_menu | Select-Object -Last 117

If the line(s) you need might be anywhere in the file, or if you’re trying to determine if a job number that ought to appear only once actually appears more than once, you’ll probably want to create an associative array, also know as a hash.

If you have a PowerShell function that can extract, say, a job number from an individual menu line, you can hash the entire file using just a couple of PowerShell commands:

$big_menu_hash = @{}        # create an empty hash
$big_menu | ForEach-Object { $big_menu_hash[Get-JobNumber( $_ )] = $_}

(if there is more than one instance of JobNumber in the file, the later instance(s) will overwrite the entries for the earlier one(s) in $big_menu_hash — you need a bit more logic than this to account for multiple appearances of a job number)

the Out-Gridview cmdlet

Maybe you don’t need to search the entire file. For example, maybe you just want to grab a couple of lines (maybe for testing before running against the entire file). Is there an alternative to opening the file in a text editor and copying the line(s) you want to another text editor window?

If you’re using PowerShell 3.0 or greater, you have access to the Out-Gridview cmdlet with the -PassThru feature. Just pipe the variable to Out-Gridview, and then use Ctrl-Click to select the rows you want:

$selected_menu_lines = $todays_menu | Out-Gridview -PassThru

Of course, you aren’t limited to piping variables to Out-Gridview – you can make a selection from a text file on disk directly:

$selected_menu_lines2 = Get-Content \\serv2\cust42\menu\MENU.TXT | Out-Gridview -PassThru

I hadn’t used it myself before today, and I was just blown away by how well it worked!

One of goals in my new approach to my job is to speed up our Quality Control (QC) processes. We use an internal website for a lot of our QC checks. This website requires manual selection of each job to be checked. It would help if we could feed a list of jobs to be QCed to a script that would automate the selection of each job.

There are a number of well-known tools to do this kind of browser automation, many of them free. However, since our QC website works about as well in Internet Explorer as any other browser, I think my best approach will be to do COM automation from PowerShell, since all the parts of the solution are built in to Windows.

Accordingly, I’ve started searching for reference material on the web:

• Use PowerShell to Check In for Flight (via Microsoft’s “Hey, Scripting Guy!” blog)
• Using IE Automation in PowerShell to Simplify Browser-Based Tasks
• Automating Internet Explorer with PowerShell
  (uses “while ($ie.Busy –eq $true) { Start-Sleep –Seconds 1; }” to wait until the browser is ready)
• IE Automation with Powershell (StackOverflow thread)
  (uses “while($ie.ReadyState –ne 4) {start–sleep 1}” to wait until browser is ready –
  I think the “$ie.Busy -eq $true” method is more intuitive, though)
• Automating Internet Explorer using PowerShell and resetting Alternate CSS URL –
  this is a nice article – the author breaks the script down into functions, and includes a function to execute jQuery and other JavaScript from the PowerShell script.