v3 isn't complete yet - did you notice the alpha tag?

• Context menu integration is coming back
• drag and drop is coming back
• HDD performance is about on par with 2.6.2 but I am going to limit the thread count to 1 for mechanical drives anyway - I take your point there actually, the more small files there are the slower a HDD will be when trying to run this. I'll fix it!
• The 16 processes you see very likely aren't all writing to the drive at once, the GC doesn't collect them instantly when the compact function is completed. Nonetheless, even a basic SATA SSD's controller is capable of at least 32 "simultaneous" operations - however I believe everything passed through Windows is handed off sequentially to the controller anyway (the simultaneous part comes from the controllers queue capabilities and not a true parallel write, SSDs batch and cache writes to minimise unnecessary amplification). Benchmarking software such as CrystalDiskMark hit it a lot harder, and because of how many process calls are made, this program is CPU limited anyway as soon as you try to compress many small files in a row. Run the native compact.exe function on a whole folder and you'll see that CompactGUI doesn't complete actions any faster than it does

It's something I've wanted to do for ages (I've been following this thread) but I hate using PInvoke especially in vb.net, there's a lot of stumbling blocks that don't exist in C# and other languages which makes it slow going.
I had a test piece built 2 years ago which kinda worked, but because it was in a scrap project file I never took the time to make it legible so when I checked it before rewriting v3, it was a mess. I'm slowly piecing it back together again.

However, you might find that v3 works almost as fast as just compressing via compact.exe anyway :). By spawning parallel processes the overhead of each process.Start() is somewhat mitigated as there's always a process (or several) actively compressing while another loads. The main bottleneck comes from rapid loading of tiny files, which v3 gets around by skipping files smaller than the disk's allocation size as they won't be compressible (by default 4kb).

I will also say Compactor's creator is a far more talented programmer using a better language too :)

Okay so I was bored today.

tl;dr

expect to see a new alpha using the Windows API soon.
I had grossly overestimated how hard it would be to use (I thought I'd have to be responsible for rewriting each file safely, which I don't have to do!)

The long version:

I just spent the last 6ish hours trying to get this working.
I thought I'd try reading the file data first, to avoid potential damage by a failed write. Seemed like a reasonable approach right? Wrong.
So, Microsoft provides this documentation for the read entry point of WofAPI. Looked pretty straightforward, so I cobbled together this DLLImport:

Rather than using all the optional out parameters, I figured I'd get the main bit working first.
Seems simple, right? Well, this is the bit that stumped me for most of the last 5 hours. At first I thought it was because the entrypoint expected an LPCWSTR instead of a regular string. But no amount of marshalling would fix the error I kept getting:

I was ready to tear my hair out.
Wait! If the file is not compressed, it works and returns 0 as expected. But if the file is compressed, I get the above error all over again.
At this point I started thinking - maybe it needs unsafe memory access (or something, what did I know) to find the sparse file?
Maybe I should write a library in C# that I can then call across?

1. I've never used C# before
2. That didn't work anyway, I couldn't even get it to work on an uncompressed file

So now I knew it wasn't working in C# either, and that's Microsoft's favourite child (not the cripple that they've left VB become).

Maybe the problem wasn't with me, but with the docs?

And here lies the problem:

You see those four unassuming, innocent words next to each parameter that says optional?

OPTIONAL MY ARSE

All that time wasted, and all I had to do was use every parameter and it works fine

And then I turned to the actual problem (so I thought) of how to safely recreate a file as sparse.
This is a literal non-issue. I was just looking at the wrong docs. That code I was scratching up two years ago? Yeah it was recreating the existing file in chunks of 4K/8K/16K/32K manually depending on which compression mode you chose, and replacing the original. Unsafe as hell. Completely unnecessary.
So now I have a working version of CompactGUI that doesn't use compact.exe, I just need to test it for safety (for example working out where the force and ignore flags come from in compact.exe

@kwencel try out the new release :)

wouldn't a dynamic system based on the entropy value of the input file be better

It would indeed, but I wouldn't know how to make that accurate, seems to be a lot of issues around that online as it is.
An alternative would be to grab a few random chunks out of each file and do a mini "test run" compression on them?

decompresses the compressed file if at a certain percentage (20, 50 ?) of compression of the file it has not reached a user-defined compression threshold.
should be doable, but actually compressing through the windows API does automatically skip files pretty quickly - it tries briefly and if a certain threshold isn't crossed it aborts the compression.

It would indeed, but I wouldn't know how to make that accurate, seems to be a lot of issues around that online as it is. An alternative would be to grab a few random chunks out of each file and do a mini "test run" compression on them?

Regarding entropy https://github.com/horsicq/Detect-It-Easy/ tool can calculate the entropy and says if the file is likely compressed,
but yes i think that a chunks compression test should be enough to reduce false positive (good file candidates).