Wednesday, January 4, 2012

Powershell (v2) - Save Objects for Offline Analysis (Report from Scripting Guys)

As I worked up a post to help address an analysis issue I had discovered in the Technet forums,
Practical folder question: how many files/folders in a folder is too many?Practical folder question: how many files/folders in a folder is too many?
I started a new thread to try and figure out how to keep from reinventing the wheel.  Instead of running Get-ChildItem over and over and over I just knew there had to be a way to save collections to disk; I just didn't know what it was.  In my second thread,
Is there a way to write a PSObject in a file/to disk?
 mjolinor points out,
Export-csv or export-clixml.

For gci, export-csv should work. For complex objects, use export-clixml. For really complex objects you may need to change the default fidelity depth.
The following link also indicates the same thing:
Learn How to Save PowerShell Objects for Offline Analysis 
Currently, I am working on a project with a folder housing over 450,000 files.  When I attempted to save to .csv and .xml, to get a feel for comparative analysis, I found Export-CliXml is much more verbose in term of object data:
PS C:\Documents and Settings\will\My Documents\Conversions> dir


Directory: C:\Documents and Settings\will\My Documents\Conversions


Mode LastWriteTime Length Name
---- ------------- ------ ----
-a--- 1/4/2012 3:55 PM 537 1. Analysis.ps1
-a--- 1/4/2012 5:33 PM 529006278 summary.csv
-a--- 1/4/2012 4:57 PM 2979435458 summary.xml
 
In raw length XML is 5.6 times larger than CSV.  When I attempt to run a Group-Object analysis on a variable created from Import-Clixml I keep getting Exceptions,
$files | Select-Object name,extension | Group-Object -Property Extension;
Exception of type 'System.OutOfMemoryException' was thrown.
At :line:0 char:0
While I like this idea in theory, I am not sure how practical it is in real usage.  More posts to follow I'm sure.

0 comments:

Post a Comment