Detecting AI Art Humans

I just made a discovery (that was actually obvious if you think how DAE works).

When you use DAE to create AI Art you get a random mix of …

a) Portaits of humans
b) Landscpaes (drawn by kids)
c) Something out of the whe wild imagination of someone on psychodelic drugs

I sorta like the 'Humans" the AI Art creates but splitting them out from the mass of random other stuff is somewhat tedious.

Here’s a run I did of 100 AI Images…

Spotting the humans, as is clearly indicated by the above, is possible but a real pain in the butt if you want to only get Humans

Now, take the random images above and run background removal on them and you get…

You will notice that in this lot of images the humans really stand out. This is because of a side effect of the way that DAE works - it’s Neural Net is set to find Humans (I keep asking for other recognisers as options but nothing ever happens - grumble…). The second lot of images is actually identical to the first in terms of colour - what makes the visual change is the opacity (alpha channel) you see in the “background removed” images. Actually what’s happened is that the alpha is set high for something that looks like a human and really low for everything else.

It is therefore possible to identify which AI Art images are “Human” simply by calculating the percentage of pixels with a high alpha. Any AI image that has > say 60% high alpha pixels is human and the rest are not.

IF DAE had a switch to change the recognizer to, for example, “Landscape” for Background Removal then the second image would show all the landscapes (PLEASE, Cedric et al - do something about this…)

1 Like

Interesting find (I was playing with deep demo version) looking at what artwork was showing never occurred to me that it was heavy on the human filter but at the same time considering this tool probably uses vgg as it’s base never had the chance to lookup the image library as you need to request for a academic access. Indeed I agree variety would be a nice touch however all depends on what was farmed to create such a database (For me no render card using online tools takes me 1h for 2000 images) coco db would take me 42h to just get such a library so indeed DAE the better bet for me (Waiting for my tax return to get it) still great job in the observation category.

1 Like

It’s VGG19 according to the filenaming

My machine can generate 1000 AI images per hour (it’s got a GPU but it can’t be used - 3060)
The background removal manage < 1/2 the speed but that’s acceptable as it can be automated.

As there are pre-defined nets that recognise other things than humans it would be possible to do other sorting, identifying landscapes would be nice.

I wrote a little test scripts in PHP CLI to automate background removal and alpha counting. The alpha count thing is very basic simply making a count based on alpha > 50% or not to create a “cover” number - just to see what came out. The results were rather interesting as they pretty much nail the Human images (with a few false positives) - results below the line…

Note the PHP used below was adapted from the background removal PHP which was itself adapted from the original image generation PHP - that’s why it’s got pointless processing speed info (which is very to the point with things that take a long time)

I really must release instructions on how to do this stuff (as easy to follow withg zero knowledge as possible is harder when the result is something the user can alter the functionality of)

The results

=================================================================

image 000001 : cover = 51.078 : time = 0.020 : avg = 0.020 sec : Rate = 3032.758 per min, 181965 per hour
image 000002 : cover = 0.000 : time = 0.017 : avg = 0.018 sec : Rate = 3281.201 per min, 196872 per hour
image 000003 : cover = 55.218 : time = 0.019 : avg = 0.019 sec : Rate = 3224.197 per min, 193451 per hour
image 000004 : cover = 55.640 : time = 0.018 : avg = 0.018 sec : Rate = 3243.248 per min, 194594 per hour
image 000005 : cover = 62.428 : time = 0.018 : avg = 0.018 sec : Rate = 3273.969 per min, 196438 per hour
image 000006 : cover = 0.000 : time = 0.019 : avg = 0.018 sec : Rate = 3257.866 per min, 195471 per hour
image 000007 : cover = 0.278 : time = 0.018 : avg = 0.018 sec : Rate = 3266.762 per min, 196005 per hour
image 000008 : cover = 53.230 : time = 0.018 : avg = 0.018 sec : Rate = 3276.971 per min, 196618 per hour
image 000009 : cover = 30.433 : time = 0.020 : avg = 0.018 sec : Rate = 3252.669 per min, 195160 per hour
image 000010 : cover = 0.000 : time = 0.019 : avg = 0.018 sec : Rate = 3243.428 per min, 194605 per hour
image 000011 : cover = 7.484 : time = 0.017 : avg = 0.018 sec : Rate = 3263.311 per min, 195798 per hour
image 000012 : cover = 12.371 : time = 0.018 : avg = 0.018 sec : Rate = 3272.957 per min, 196377 per hour
image 000013 : cover = 0.000 : time = 0.019 : avg = 0.018 sec : Rate = 3265.148 per min, 195908 per hour
image 000014 : cover = 0.024 : time = 0.019 : avg = 0.018 sec : Rate = 3258.776 per min, 195526 per hour
image 000015 : cover = 65.666 : time = 0.019 : avg = 0.018 sec : Rate = 3252.592 per min, 195155 per hour
image 000016 : cover = 0.000 : time = 0.019 : avg = 0.018 sec : Rate = 3243.437 per min, 194606 per hour
image 000017 : cover = 0.000 : time = 0.016 : avg = 0.018 sec : Rate = 3269.445 per min, 196166 per hour
image 000018 : cover = 0.000 : time = 0.020 : avg = 0.018 sec : Rate = 3254.508 per min, 195270 per hour
image 000019 : cover = 0.614 : time = 0.019 : avg = 0.018 sec : Rate = 3246.407 per min, 194784 per hour
image 000020 : cover = 36.209 : time = 0.019 : avg = 0.019 sec : Rate = 3242.294 per min, 194537 per hour
image 000021 : cover = 0.000 : time = 0.019 : avg = 0.019 sec : Rate = 3236.003 per min, 194160 per hour
image 000022 : cover = 0.000 : time = 0.018 : avg = 0.019 sec : Rate = 3237.474 per min, 194248 per hour
image 000023 : cover = 30.210 : time = 0.018 : avg = 0.018 sec : Rate = 3243.469 per min, 194608 per hour
image 000024 : cover = 0.000 : time = 0.016 : avg = 0.018 sec : Rate = 3259.414 per min, 195564 per hour
image 000025 : cover = 7.067 : time = 0.019 : avg = 0.018 sec : Rate = 3253.364 per min, 195201 per hour
image 000026 : cover = 94.681 : time = 0.020 : avg = 0.018 sec : Rate = 3244.847 per min, 194690 per hour
image 000027 : cover = 0.000 : time = 0.019 : avg = 0.019 sec : Rate = 3240.932 per min, 194455 per hour
image 000028 : cover = 8.763 : time = 0.018 : avg = 0.018 sec : Rate = 3247.260 per min, 194835 per hour
image 000029 : cover = 0.000 : time = 0.018 : avg = 0.018 sec : Rate = 3252.407 per min, 195144 per hour
image 000030 : cover = 0.000 : time = 0.020 : avg = 0.019 sec : Rate = 3240.759 per min, 194445 per hour
image 000031 : cover = 52.203 : time = 0.018 : avg = 0.018 sec : Rate = 3245.199 per min, 194711 per hour
image 000032 : cover = 43.022 : time = 0.019 : avg = 0.019 sec : Rate = 3241.269 per min, 194476 per hour
image 000033 : cover = 64.825 : time = 0.018 : avg = 0.018 sec : Rate = 3244.150 per min, 194648 per hour
image 000034 : cover = 0.000 : time = 0.018 : avg = 0.018 sec : Rate = 3245.955 per min, 194757 per hour
image 000035 : cover = 0.000 : time = 0.017 : avg = 0.018 sec : Rate = 3253.793 per min, 195227 per hour
image 000036 : cover = 0.000 : time = 0.019 : avg = 0.018 sec : Rate = 3249.207 per min, 194952 per hour
image 000037 : cover = 0.000 : time = 0.019 : avg = 0.018 sec : Rate = 3247.748 per min, 194864 per hour
image 000038 : cover = 0.000 : time = 0.018 : avg = 0.018 sec : Rate = 3247.731 per min, 194863 per hour
image 000039 : cover = 64.995 : time = 0.018 : avg = 0.018 sec : Rate = 3252.116 per min, 195126 per hour
image 000040 : cover = 0.000 : time = 0.017 : avg = 0.018 sec : Rate = 3257.379 per min, 195442 per hour
image 000041 : cover = 36.577 : time = 0.019 : avg = 0.018 sec : Rate = 3256.951 per min, 195417 per hour
image 000042 : cover = 0.000 : time = 0.019 : avg = 0.018 sec : Rate = 3254.790 per min, 195287 per hour
image 000043 : cover = 49.630 : time = 0.020 : avg = 0.018 sec : Rate = 3250.174 per min, 195010 per hour
image 000044 : cover = 0.000 : time = 0.018 : avg = 0.018 sec : Rate = 3251.321 per min, 195079 per hour
image 000045 : cover = 48.989 : time = 0.018 : avg = 0.018 sec : Rate = 3251.615 per min, 195096 per hour
image 000046 : cover = 66.920 : time = 0.017 : avg = 0.018 sec : Rate = 3255.390 per min, 195323 per hour
image 000047 : cover = 0.000 : time = 0.018 : avg = 0.018 sec : Rate = 3255.890 per min, 195353 per hour
image 000048 : cover = 0.000 : time = 0.017 : avg = 0.018 sec : Rate = 3260.450 per min, 195626 per hour
image 000049 : cover = 0.000 : time = 0.019 : avg = 0.018 sec : Rate = 3258.615 per min, 195516 per hour
image 000050 : cover = 0.002 : time = 0.017 : avg = 0.018 sec : Rate = 3264.212 per min, 195852 per hour
image 000051 : cover = 53.374 : time = 0.017 : avg = 0.018 sec : Rate = 3267.290 per min, 196037 per hour
image 000052 : cover = 42.749 : time = 0.018 : avg = 0.018 sec : Rate = 3267.847 per min, 196070 per hour
image 000053 : cover = 0.000 : time = 0.018 : avg = 0.018 sec : Rate = 3269.556 per min, 196173 per hour
image 000054 : cover = 1.012 : time = 0.020 : avg = 0.018 sec : Rate = 3264.266 per min, 195855 per hour
image 000055 : cover = 0.804 : time = 0.018 : avg = 0.018 sec : Rate = 3264.663 per min, 195879 per hour
image 000056 : cover = 7.028 : time = 0.015 : avg = 0.018 sec : Rate = 3273.855 per min, 196431 per hour
image 000057 : cover = 0.000 : time = 0.019 : avg = 0.018 sec : Rate = 3272.777 per min, 196366 per hour
image 000058 : cover = 0.000 : time = 0.018 : avg = 0.018 sec : Rate = 3272.799 per min, 196367 per hour
image 000059 : cover = 9.832 : time = 0.019 : avg = 0.018 sec : Rate = 3269.568 per min, 196174 per hour
image 000060 : cover = 38.290 : time = 0.021 : avg = 0.018 sec : Rate = 3262.483 per min, 195748 per hour
image 000061 : cover = 0.000 : time = 0.017 : avg = 0.018 sec : Rate = 3266.163 per min, 195969 per hour
image 000062 : cover = 17.624 : time = 0.018 : avg = 0.018 sec : Rate = 3266.703 per min, 196002 per hour
image 000063 : cover = 0.000 : time = 0.018 : avg = 0.018 sec : Rate = 3268.031 per min, 196081 per hour
image 000064 : cover = 47.142 : time = 0.017 : avg = 0.018 sec : Rate = 3270.986 per min, 196259 per hour
image 000065 : cover = 27.332 : time = 0.018 : avg = 0.018 sec : Rate = 3271.511 per min, 196290 per hour
image 000066 : cover = 0.000 : time = 0.020 : avg = 0.018 sec : Rate = 3266.114 per min, 195966 per hour
image 000067 : cover = 37.886 : time = 0.018 : avg = 0.018 sec : Rate = 3267.246 per min, 196034 per hour
image 000068 : cover = 0.000 : time = 0.018 : avg = 0.018 sec : Rate = 3268.081 per min, 196084 per hour
image 000069 : cover = 0.000 : time = 0.018 : avg = 0.018 sec : Rate = 3269.737 per min, 196184 per hour
image 000070 : cover = 0.000 : time = 0.019 : avg = 0.018 sec : Rate = 3268.134 per min, 196088 per hour
image 000071 : cover = 0.000 : time = 0.019 : avg = 0.018 sec : Rate = 3266.879 per min, 196012 per hour
image 000072 : cover = 0.000 : time = 0.018 : avg = 0.018 sec : Rate = 3266.595 per min, 195995 per hour
image 000073 : cover = 0.793 : time = 0.020 : avg = 0.018 sec : Rate = 3263.713 per min, 195822 per hour
image 000074 : cover = 8.763 : time = 0.016 : avg = 0.018 sec : Rate = 3268.396 per min, 196103 per hour
image 000075 : cover = 0.000 : time = 0.017 : avg = 0.018 sec : Rate = 3272.782 per min, 196366 per hour
image 000076 : cover = 0.000 : time = 0.019 : avg = 0.018 sec : Rate = 3272.085 per min, 196325 per hour
image 000077 : cover = 61.842 : time = 0.018 : avg = 0.018 sec : Rate = 3273.599 per min, 196415 per hour
image 000078 : cover = 0.000 : time = 0.017 : avg = 0.018 sec : Rate = 3277.621 per min, 196657 per hour
image 000079 : cover = 0.000 : time = 0.017 : avg = 0.018 sec : Rate = 3281.199 per min, 196871 per hour
image 000080 : cover = 0.000 : time = 0.021 : avg = 0.018 sec : Rate = 3276.035 per min, 196562 per hour
image 000081 : cover = 39.191 : time = 0.017 : avg = 0.018 sec : Rate = 3279.127 per min, 196747 per hour
image 000082 : cover = 0.000 : time = 0.018 : avg = 0.018 sec : Rate = 3280.637 per min, 196838 per hour
image 000083 : cover = 0.000 : time = 0.017 : avg = 0.018 sec : Rate = 3283.694 per min, 197021 per hour
image 000084 : cover = 18.299 : time = 0.017 : avg = 0.018 sec : Rate = 3286.682 per min, 197200 per hour
image 000085 : cover = 28.601 : time = 0.018 : avg = 0.018 sec : Rate = 3287.189 per min, 197231 per hour
image 000086 : cover = 0.000 : time = 0.018 : avg = 0.018 sec : Rate = 3288.032 per min, 197281 per hour
image 000087 : cover = 0.000 : time = 0.018 : avg = 0.018 sec : Rate = 3288.216 per min, 197292 per hour
image 000088 : cover = 0.000 : time = 0.018 : avg = 0.018 sec : Rate = 3288.955 per min, 197337 per hour
image 000089 : cover = 0.000 : time = 0.019 : avg = 0.018 sec : Rate = 3287.840 per min, 197270 per hour
image 000090 : cover = 0.000 : time = 0.018 : avg = 0.018 sec : Rate = 3287.650 per min, 197258 per hour
image 000091 : cover = 0.000 : time = 0.016 : avg = 0.018 sec : Rate = 3291.662 per min, 197499 per hour
image 000092 : cover = 0.000 : time = 0.018 : avg = 0.018 sec : Rate = 3292.059 per min, 197523 per hour
image 000093 : cover = 0.000 : time = 0.016 : avg = 0.018 sec : Rate = 3295.667 per min, 197740 per hour
image 000094 : cover = 35.831 : time = 0.017 : avg = 0.018 sec : Rate = 3297.571 per min, 197854 per hour
image 000095 : cover = 39.550 : time = 0.017 : avg = 0.018 sec : Rate = 3299.952 per min, 197997 per hour
image 000096 : cover = 0.576 : time = 0.020 : avg = 0.018 sec : Rate = 3297.114 per min, 197826 per hour
image 000097 : cover = 0.000 : time = 0.016 : avg = 0.018 sec : Rate = 3300.850 per min, 198051 per hour
image 000098 : cover = 0.173 : time = 0.019 : avg = 0.018 sec : Rate = 3299.824 per min, 197989 per hour
image 000099 : cover = 0.000 : time = 0.019 : avg = 0.018 sec : Rate = 3298.827 per min, 197929 per hour
image 000100 : cover = 60.610 : time = 0.017 : avg = 0.018 sec : Rate = 3300.871 per min, 198052 per hour
1 Like

Impressive I’m quite the coder myself when it goes to dos batch files and scripting in practically any language as for releasing something for others a good chunk of people wouldn’t grasp the concept one thing I noticed is that the generate AI Artwork ain’t local by accident (My internet had crashed) and the option was asking for a internet connection so I’m wondering if it’s not part of a per-defined list either way the results do give back some great loop backs for making new abstract art. Only feature I noticed that annoys me and seems that it was mentioned on the forum of a possible fix for sharing styles besides that yes the GPU is a pain but there are not many current options besides NVIDIA cuda and there is no fake emulator to use such as tensorflow-directml for python to speed things up but I cannot complain AI / Machine learning is still in early phases and a tool like this for that price you just can’t go wrong. Well Peardox keep it up I’m fascinated at the stuff your discovering.

When any DAE product starts it checks what license you’ve got and if it’s valid - for that it needs Internet. My scripts start DAE CLI every few seconds so it checks my license every few seconds, I have noted this to DAE privately.

Tensorflow 2 is in development (first informed in Dec by DAE management, still waiting - patiently - it gets here when it gets here, I can wait)

I’m just touching up the version of my PHP that automates the creation of AI images so that it is at least understandable to someone who knows how to wrote code (almost done - just added an ETA to it so you have a rough idea how long it’s going to take)

I then plan to write a blog article on how to install PHP, DAE CLI and my script so people can do this stuff for theirselves (I’ve been promisong this for ages - about time I did it). I also plan to supplement it with the above mentioned scripts which will prove instructional if people want to do similar but different things with the CLI.

Finally I’ll write one that takes all the parts and wraps it up in one script that just generates “Humans” - this will actually be a simple case of write an test image, remove background, check “Humanity” and finally throw it away or save it as useful output.

There’s actually a huge amount of things you can do with the CLI and some creative scripting.

I’m using PHP because…

  1. I know it
  2. It works on web servers (meaning I can deploy to a server if I want)
  3. It offers a host of features including image processing (vital for testing alpha)
  4. It’s not Python (the Internet is awash with some truly dreadful Python)
  5. It’s relatively easy for a novice to install (unzip an archive and possibly edit one file)

Finished script - lets make 1 million images …

Oops, no PC for six weeks, sod that for an idea…:slight_smile:

Actually I did create 100,000 on a server

2 Likes

So a 100K new humans on the Russian server then? Or are you not on to the automated discard yet.
Thanks for all you do with this product. Once I can leverage my 3060 I will be licensing DAE - for now it feels like running with both hands (many CPUs) tied behind my back.

Yup, Russia but as some idiot started a War I’ve not tried it in a while.

I’ve got a normal (UK-based) small linux server so I just ran stuff on the Russian box then offloaded the results to the UK server.

I THEN discovered that the UK server was more than happy to run DAE but owing to having very little memory only for small images (like AI Art)

Actually CLI don’t use the GPU in the first place with stock DAE so I’m not using my own 3060 ATM anyway

It will be interesting to override the Tensorflow DLL with a 3060 compatible one compiled specifically for my CPU when they finally release TF2

Something else that’ll be worth trying is compiling TF2 with RocM support and finding someone witrh a decent ATI card to see if that works.

Technically I guess my laptop may support RocM as the default integrated GPU is ATI - the ‘proper’ GPU is the 3060 - I’ll have to go read up on this…

So a 100K new humans on the Russian server then?

Err, actually 100k assorted AI images.

I do now have a way to work out which ones are human - see last image on …
https://peardox.com/2022/04/15/bulk-background-removal/

Finally purchased the life license today (Go figure I’d be lucky and get the Easter special) I’ll be able to share some tips and style tricks now that everything is fully functional and should I find suggestions on improvements I’ll definitely post them in the correct section.

Quick question for you to use the CLI do I need to disable the desktop one first or it will act as activated? Edit: I tested it it does activate both together. I tried the generator using %random% for the filenames in dos works like a charm.

Didn’t use my Ad link to buy :frowning: Oh well… I guess one of the problems about having ad links for something people already have is that they tend not to buy it again :slight_smile:

The license is on the machine so same machine = same license.

You can actually have a CLI session running and a GUI one at the same time - not overly recommended though as both will want to write to the debug log.

If you want to use your license on a different machine you log in to your account on the main DAE site and de-activate your license. Then on the other machine you start the GUI and enter your license key so the license is then on that machine.

This, of course, means that you can’t use DAE on the machine you transferred your license from to do get it back just de-activate it again and activate on the original machine.

I’ve run DAE on have half a dozen machines and/or OSes switching between them as detailed above without problems. As you’ll have guessed, it even works on a Russian Cloud PC (or will when someone stops some bloody stupid war)

I do occassionally run both Linux + Windows on the same box so I’ve even swpapped the license out and in on the same machine :slight_smile:

Back to removing backgrounds…

Thanks for the info and indeed you are correct about both in use at the same time as for using your site link I just didn’t think about it noticed the promotion and had used the direct link to get the promo code. Should I get more people interested in DAE I’ll send them to your referral page. Back on track to your creations quite impressive on the results only thing that annoys me a bit is that the application opens and closes for each process a bit of a extra load but I guess that’s the best option at the time that CLI was created.

the application opens and closes for each process

How do you mean? I ask because some of my tests resulted in the DAE logo appearing but that shouldn’t happen with the scripts I’ve written.

Basically you can start DAE in such a way that a CLI will run and you’ll get a Logo splash but if you follow the directions in my posts this shouldn’t happen.

If you’re referring to the fact that every run involves a separate launch of DAE CLI then, yes, that’s unavoidable (and, believe me, I’ve tried avoiding it).

The fact remains that even with the repeated loading of CLI the automated tasks are immensely faster than the alternative (lots of mouse clicking)

I’ll have to write a little test to time batch conversion via CLI vs batch conversion via GUI. I already know that GUI will be faster in this test as it only loads DAE once while I’ll have to do it 121 or 2 times (* daily * can’t be used from CLI). The results will be interesting and give an accurate figure for the overhead.

Actually I want to write a batch script anyway - some guy asked specifically how you could process a bunch of images using only a few selected filters so it’s a worthwhile little utility anyway.

I have long planned to write a GUI that does what the scripts do - the scripts are useful tests for a GUI version (I acutally started one well over a year ago but gave up on it cos it looked awful)

60 background removals left for this run, only 60k more to do…

@Peardox " every run involves a separate launch of DAE " that’s what I was trying to explain but I’ve solved that issue using a ram drive including a bug fix that I posted in regards to the java hick ups as for the daily indeed it cannot be used that way however I’m sure that like any model file if I use a placer (Ie: Replace one with the model of the daily for batch runs it would probably work) but most definitely I’m using Autohotkey Gui gen to make a interface with command options such as location, amounts I want to render and stacking styles I could use something more complex but for my needs this will do fine. Like you said the Automated tasks that we can do via command line are a time saver.

Ooh, finally got it working properly in Linux :slight_smile:

It’s half the speed of my new lappy but I can run it 24/7 so in the long run it’ll finish sooner even if it is slower.

The Ubuntu box is my old Laptop (2015 Dell that was decent at the time). There were a few obstacles but worked them out and it’s running nicely now.

1 Like

In three hour from now I’ll have 100k AI images with their backgrounds all removed.

Tying this to some other analysis that works out colour, darkness and contrast numbers I should be able to sort of classify the more interesting images fairly easily (that’s a quick run into a DB though). Wondering if I want to classify hue ATM - might prove useful

1 Like

Do a search for “Sort images by color with ImageSorter” I’m working on something similar but more recent as this program is real old to do exactly that to be able to manage libraries of images to make my own model pack using other tools but I love the hue idea i’ll test it out with imagemagick see what I can pull off checking the hue levels with some dos batch files.

Will do

For colour ATM I’m just creating a number based on 512 palette to identify the colours. Actually, cool idea, I could alter it to sort the palette which would give me a better ‘how numch colour in in this image’ number.

I like the idea of a hue number as that way I can group all the mostly red images together for example.

There’s also Perceptual Hashing - I like the idea of this one too. This one is actually one of the techniques used in all AI image stuff anyway.

When all 100k have finished and I’ve analysed them all I’ll do a little page that selects images based on those criteria to see how it all works. Haven’t got a clue what the results will be but it’ll be intteresting to find out what happens. If it’s good I might make the results public - and plaster ads all over it :slight_smile:

1 Like

I just typed “php hue” into google

It came back with a zillion ads for Philips Light Bulbs :frowning:

1 Like

@Peardox Yup google at it again search for something get something completely unrelated as for Perceptual hashing about the same method used in YouTube’s algorithm for finding copyrighted stuff so indeed that might be the best approach as for everything else if we don’t try it we’ll never know if it works or not good luck with that. :grin: