Help needed: Program or configure a web / XML crawler

Those changes should be fine, they're working OK for me using Powershell 4.0

It turns out that the Invoke-WebRequest cmdlet was introduced in PowerShell version 3.0 If you getting an error then it sounds like you may have an earlier version. Though it also sounds like you're using the Powershell ISE so I thought that meant you're using version 3.0 at least. Colour me a little confused.

You can type $psversiontable to find your version.

You can upgrade to the latest version here

http://www.microsoft.com/en-gb/download/details.aspx?id=40855

Let me know how it pans out.

Yes, version 2.0, and yes, PowerShell ISE :D

I'll try the update and report later what happens.
 
Ok, I have PowerShell 4.0 now, script apparently works as designed, I do get CSV files filled with n lines of data.

Same problems as I have with some of the XSL tools:

a) Foreign characters are masked as "??"
b) (This is by your design:) Script doesn't open the URLs listed in each data line to retrieve additional fields there - but that crawling of referenced documents is why I came here in the first place. I can get most of the other data (license info is missing) from parsing the HTML as displayed by ae911truth.org. But I need to dig into the person files to retrieve their Bio and Personal 9/11 Statement.
 

ISF - Join now!

Every member here is approved by hand. No bots, no spam, just people who care about evidence and honest debate.

Membership is free!

Create your free account

Back
Top Bottom