Sorry your browser is not supported!

You are using an outdated browser that does not support modern web technologies, in order to use this site please update to a new browser.

Browsers supported include Chrome, FireFox, Safari, Opera, Internet Explorer 10+ or Microsoft Edge.

Geek Culture / Resizing an array vs more iterations, which is better?

Author
Message
Phaelax
DBPro Master
19
Years of Service
User Offline
Joined: 16th Apr 2003
Location: Metropia
Posted: 5th Feb 2021 16:37
A script I'm working on in powershell (which I'm brand new to) and trying to determine which would be better, removing elements from an array list or just setting those elements to null instead?

I'm basically doing a sync on user info. Given an arraylist 'B' of 12k items, I iterate over them and compare them to an existing list of roughly the same size, call it list 'A'. If I find a match of B in list A, update the existing object in A to match the new data from B then remove from A. Anything left in A after removing all of B from it, I'll need to process in another task.

I expect both lists to be somewhat close in size, so A would be fairly small by the end leaving me with a small list to iterate over for the final task. How efficiently arrayList works in powershell I don't know. If it recreates the list of a new size and copies everything or what. The alternative would be instead of removing the elements I would just NULL that object. Then my final process loops over the 12k elements again and looks for the non-null objects to process. Would the latter be faster in the end?

Hope this made some sense.
Tiled TMX Importer V.2
XML Parser V.2
Base64 Encoder/Decoder
Purple Token - Free online hi-score database
Legend of Zelda

"I like offending people, because I think people who get offended should be offended." - Linus Torvalds
Raven
17
Years of Service
User Offline
Joined: 23rd Mar 2005
Location: Hertfordshire, England
Posted: 11th Mar 2021 09:27
List[ ] in PowerShell works IDENTICAL to how it does in C#., essentially this is identical to std::vector< > in C/C++
In essence what it does is hides the Complexity of creating a New Array then filling it with all the existing data and updating the pointer references for you... but you're still essentially using a Static Array.
As such Adding / Removing Elements does come at a (noticeable) performance hit.

Now the beauty of Contiguous Arrays as opposed to a Linked List Array (which is an Array consisting of Structure that points to the Previous and Next Element) is that they're exceptionally fast to perform Sequential Operations on., such-as Search and Sort., but this comes at the cost that there is overhead when you Expand or Contract them.

What I'd recommend doing is setting the Values to "Null", that you no longer need... Sort the Array; then when you have a certain number of Null Objects (say 10) you can then resize the Array to just remove them.
Remember that you can use Operators (+ - += -=) on Arrays rather than the Elements to perform operations.

So $myArray -= 10 would remove the last 10 Elements and create said new Array for you in a single operation (rather than individual elements); and this works out faster.

Login to post a reply

Server time is: 2022-08-18 12:43:52
Your offset time is: 2022-08-18 12:43:52