Out of disk space?

I just ran out of disk space! And the culprit is DAE…

OK, slightly my own fault as I know this issue exists plus the fact that I’m testing out scripts means I’m running thousands of DAE sessions but…

Every time DAE runs it creates a temporary folder with a copy of one 75Mb file in it. The folder names are semi-random but always start with “tensorflow_native_libraries-” followed by some numbers etc

As long as DAE isn’t currently running these files are just taking up space so every so often you want to delete them (until DAE hopefully tidy up after theirselves). To do this cut & paste the following “magic” name into Windows Explorer’s address bar …

%userprofile%\AppData\Local\Temp

This will become something like C:\Users\Simon\AppData\Local\Temp (your name will replace mine) - note that this directory is not usually visible so this is one of the few ways to navigate to it.

Once you’re there just shift-select all the tensorflow_… and delete them

[DAE - Please resolve this rather old issue in 1.2.8]

It should be noted that having a LOT of files in a directory can slow things down. As I had many thousands of tensorflow_… directories clogging up my own temp dir the end result was the degredation of the speed my script was running at. With all this crap it could only manage about 600 background removal tasks per hour. Deleting the junk has increased performance and I’m up to 1300 operations an hour (amazing, eh?)

Having spotted this issue I’m going to make some alterations to the series of PHP scripts I’m writing presently to periodically clear out the rubbish.

1 Like

That’s for the warning the solution I’ve found is that I took a copy of the tensorflow_jni.dll and copied it inside the folder C:\Users<Your User Name>\AppData\Roaming\DeepArtEffects then relauch DAE and the issue will be gone (I’m assuming the same thing to do for the CLI version just place it where the EXE is) hope your drive wasn’t a SSD that’s some serious wear and tear.

Edit: On a side note I think there may be a few other files that we may need to copy into the main folder to avoid a re-creation all the time (I’ll look into it)

Yep, one SSD somewhat confused by the state of affairs.

The Roaming thing didn’t work for CLI but I know that if you put the JNI somewhere in the path then it takes precedence - this is how I got CUDA 10 to work actually.

A side effect I just confirmend was that sticking the JNI in the path also stops the temp problem.

Putting it in the program directory can be a bit hit + miss as technically the program you run is java.exe (on Linux + Mac I sometimes invoke DAE with Java from the command line)

I tried copying the JNI to other ‘usual suspect’ directories but none of those approaches resolved things

When loading a dll the order is program dir, then other places like system and finally path

So, while I can now solve my own system if I write stuff for others to use then cleaning temp regularly is the only viable option

Note By copying tensorflow_jni.dll to path my current remove 10,000 backgrounds task has increased in speed by about 6% (only 7 hours left for the remining 5593 images…)

That’s great news on the time reduction it’s amazing what a few tweaks will do in the long run for one system as for others each system is different indeed myself I have a path index that would cause nightmares to most programmers but it works so I cannot complain. Good luck with the background removals I’m sure you’ll end up with something quite surprising as for me I’ll pickup my life license on tuesday when the banks clear can’t wait only downfall I see with DAE is the lack of saving the styles when doing either random or importing (It’s always using the same filename) I could easy sell this program to the masses if I had a import export model feature so I could make a share community I figured a ugly loophole to do it using a batch file converted as a gui but like the dll fix it’s not made for everyone.

I plan to write the background removal article today so it’s ready for the W/E (my website gets more hits on Saturday). The background removal thing does directory recursion so you just give it an input + output dir and leave it alone.

One thing I noted was that as I have 100k images this would take too long to run in one sitting so I made the thing re-startable. As a result I’ve removed the background from 20k images this week over a few ‘While-I’m-Asleep’ sessions (I keep forgetting to start it when I nap - could have done way more by now)

Next week I plan to show how to work out the alpha-cover - that bits easy.

I’m using PHP for mine as it’s more complete than batch files and can be easily converted to an EXE if you really want to but the main reason is to allow the reader to play with stuff I write.

One issue with CLI is that there’s an forced load/unload delay - this takes up to about 2 seconds.

DeepArtEffectsCli.exe is actually a self extracting exe (right click on it in WinRar) so every run it has to extract itself, load Java then load the DAE code BEFORE it can actually do anything. Extracting the Java doesn’t make things any faster either although it was useful getting DAE working on my Pi and Nano boards.

I had planned to write a front-end to CLI, I’ll prob get around to that eventually…