|
Yeah sorry - that should have been double back slashes, but single forwards work too.
Can you try going to localhost:5000 to open up the dashboard and let me know if the log output shows anything that looks like an error? You can also click on the Explorer (top centre button of the dashboard) to launch the senseAI explorer, and that will allow you to test the face registration / recognition / listing.
Any info you can provide is helpful.
cheers
Chris Maunder
|
|
|
|
|
Here's what I'm seeing from the startup logs, nothing jumps out as an error:
10:06:52 AM: App directory C:\Program Files\CodeProject\SenseAI
10:06:52 AM: Started Background Removal
10:06:52 AM: Not starting Legacy Object Detection: Not set as enabled
10:06:52 AM: Started Portrait Filter
10:06:52 AM: Started Sentiment Analysis
10:06:53 AM: PortraitFilter.dll: Starting inference session...
10:06:54 AM: PortraitFilter.dll: info: Microsoft.Hosting.Lifetime[0]
10:06:54 AM: PortraitFilter.dll: info: Microsoft.Hosting.Lifetime[0]
10:06:54 AM: PortraitFilter.dll: info: Microsoft.Hosting.Lifetime[0]
10:06:54 AM: ObjectDetector.dll: info: Microsoft.Hosting.Lifetime[0]
10:06:54 AM: ObjectDetector.dll: info: Microsoft.Hosting.Lifetime[0]
10:06:54 AM: ObjectDetector.dll: info: Microsoft.Hosting.Lifetime[0]
10:06:54 AM: SentimentAnalysis.dll: 2022-06-21 10:06:54.821194: I tensorflow/cc/saved_model/reader.cc:43] Reading SavedModel from: C:\Program Files\CodeProject\SenseAI\AnalysisLayer\SentimentAnalysis\sentiment_model
10:06:54 AM: SentimentAnalysis.dll: 2022-06-21 10:06:54.827749: I tensorflow/cc/saved_model/reader.cc:148] Reading SavedModel debug info (if present) from: C:\Program Files\CodeProject\SenseAI\AnalysisLayer\SentimentAnalysis\sentiment_model
10:06:54 AM: SentimentAnalysis.dll: To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
10:06:54 AM: SentimentAnalysis.dll: 2022-06-21 10:06:54.896526: I tensorflow/cc/saved_model/loader.cc:283] SavedModel load for tags { serve }; Status: success: OK. Took 75320 microseconds.
10:06:54 AM: scene.py: Scene Detection module started.
10:06:55 AM: Latest version available is 1.4-Beta
10:06:55 AM: Version check: This is the latest version
10:06:55 AM: PortraitFilter.dll: Background Portrait Filter Task Started.
10:06:55 AM: SentimentAnalysis.dll: info: Microsoft.Hosting.Lifetime[0]
10:06:55 AM: SentimentAnalysis.dll: info: Microsoft.Hosting.Lifetime[0]
10:06:55 AM: SentimentAnalysis.dll: info: Microsoft.Hosting.Lifetime[0]
10:06:55 AM: SenseAI Object Detection module started.
10:06:55 AM: face.py: Face Detection module started.
10:06:56 AM: sense_rembg_adapter.py: RemoveBackground module started.
10:06:56 AM: textsummary.py: TextSummary module started.
10:06:56 AM: SentimentAnalysis.dll: info: SentimentAnalysis.SentimentAnalysisWorker[0]
10:06:56 AM: Sentiment Analysis module started.
Also, just to be clear, face recognition is working great (and fast!) for me:
piece = <PIL Image Object>
buf = io.BytesIO()
piece.save(buf, format='JPEG')
byte_im = buf.getvalue()
response = requests.post("http://daedelus.lan:5000/v1/vision/detection", files={"image": byte_im}).json()
for object in response["predictions"]:
print("{} - {}%".format(object["label"], round(object["confidence"]*100, 1)))
print(response)
response = requests.post("http://daedelus.lan:5000/v1/vision/face", files={"image": byte_im}).json()
print('FACE finder: {}'.format(response))
for object in response["predictions"]:
face_boxes = [object["x_min"], object["y_min"], object["x_max"], object["y_max"]]
print("{} - {}%".format(face_boxes, round(object["confidence"]*100, 1)))
face = piece.copy().crop((face_boxes[0], face_boxes[1], face_boxes[2], face_boxes[3]))
face.save(buf, format='JPEG')
byte_im = buf.getvalue()
response = requests.post("http://daedelus.lan:5000/v1/vision/face/recognize", files={"image": byte_im}).json()
print('FACE recognizer: {}'.format(response))
for object_f in response["predictions"]:
print("FACE: {} - {}%".format(object_f["userid"], round(object_f["confidence"]*100, 1)))
retults:
person - 92.3%
tennis racket - 38.7%
car - 92.5%
potted plant - 34.1%
tennis racket - 52.7%
{'predictions': [{'label': 'person', 'confidence': 0.9226199, 'y_min': 662, 'x_min': 547, 'y_max': 1522, 'x_max': 1025}, {'label': 'tennis racket', 'confidence': 0.38740522, 'y_min': 793, 'x_min': 538, 'y_max': 895, 'x_max': 771}, {'label': 'car', 'confidence': 0.924697, 'y_min': 29, 'x_min': 17, 'y_max': 509, 'x_max': 820}, {'label': 'potted plant', 'confidence': 0.34054393, 'y_min': 1353, 'x_min': 393, 'y_max': 1550, 'x_max': 652}, {'label': 'tennis racket', 'confidence': 0.5270328, 'y_min': 897, 'x_min': 270, 'y_max': 1260, 'x_max': 808}], 'success': True}
FACE finder: {'success': True, 'predictions': [{'confidence': 0.6121846437454224, 'x_min': 829, 'y_min': 682, 'x_max': 890, 'y_max': 778}]}
[829, 682, 890, 778] - 61.2%
FACE recognizer: {'success': True, 'predictions': [{'confidence': 0.6947708129882812, 'userid': 'Jasmine', 'x_min': 829, 'y_min': 682, 'x_max': 890, 'y_max': 778}]}
FACE: Jasmine - 69.5%
It's just not returning the face information through BlueIris when I point BI at the SenseAI url. It is returning all the results of the detection predictions. I don't have enough insight to see what BI is calling internally and whether the APIs match.
Thanks again for your attention!
|
|
|
|
|
Hey guys congrats on the project it's excellent.
Seems it's fully compatible with Agent DVR - might want to add that to the compatibility docs with a setup walkthrough?
Agent DVR DeepStack AI/ senseAI[^]
|
|
|
|
|
That's fantastic! I'll try it out and add it to the docs.
Thanks for the tip. 👍
cheers
Chris Maunder
|
|
|
|
|
|
Funny you should mention that! We were talking about exactly this today and I'd like it to be done this week.
I have a personal need of a racoon detector. It's a long, painful story.
cheers
Chris Maunder
|
|
|
|
|
Great, I also am working on a Python rest api that uses DeepStack to do ALPR, do you think there is a way to add it to this project. I am not a programmer but with Google I was able to make it work.
|
|
|
|
|
I see 365 scene types and 80 objects can be detected. Is there a list of detectable scenes and objects?
|
|
|
|
|
I think they are using YOLOv5 models which has the below objects
'person', 'bicycle', 'car', 'motorcycle', 'airplane', 'bus', 'train', 'truck', 'boat', 'traffic light',
'fire hydrant', 'stop sign', 'parking meter', 'bench', 'bird', 'cat', 'dog', 'horse', 'sheep', 'cow',
'elephant', 'bear', 'zebra', 'giraffe', 'backpack', 'umbrella', 'handbag', 'tie', 'suitcase', 'frisbee',
'skis', 'snowboard', 'sports ball', 'kite', 'baseball bat', 'baseball glove', 'skateboard', 'surfboard',
'tennis racket', 'bottle', 'wine glass', 'cup', 'fork', 'knife', 'spoon', 'bowl', 'banana', 'apple',
'sandwich', 'orange', 'broccoli', 'carrot', 'hot dog', 'pizza', 'donut', 'cake', 'chair', 'couch',
'potted plant', 'bed', 'dining table', 'toilet', 'tv', 'laptop', 'mouse', 'remote', 'keyboard', 'cell phone',
'microwave', 'oven', 'toaster', 'sink', 'refrigerator', 'book', 'clock', 'vase', 'scissors', 'teddy bear',
'hair drier', 'toothbrush'
|
|
|
|
|
I don't see the keyword "package". That's a bummer for my camera watching the front porch.
But hey... Detecting a toilet can be good.
|
|
|
|
|
Yeah - the amazon package delivery is a custom model we will add (or even better: source from someone kind!)
It's high on my TODO list.
cheers
Chris Maunder
|
|
|
|
|
Chris,
I could look into making a package model or add it to one of my custom models.
Mike
|
|
|
|
|
Yes please!
cheers
Chris Maunder
|
|
|
|
|
What model format are you using from what I can tell it is ONNX?
|
|
|
|
|
We're going to lean on Deepstack's model training, which is essentially the YOLO training (same code more or less) which outputs PyTorch .pt files.
cheers
Chris Maunder
|
|
|
|
|
Great, that is how all my models are trained. I just trained a package model, it only took 10 minutes on my RTX 3090. I will see how it performs before I added it to my other models.
|
|
|
|
|
Nice! Let us know how it goes.
cheers
Chris Maunder
|
|
|
|
|
Do you have your models posted somewhere? Or a quick walkthrough (or pointer) for training a model and putting it in the right format/place?
I've got all my security cameras wired up and sending alerts and it's awesome... next step is having a "package on the porch" model or a "black car is in driveway" model or a "fence gate is open" model, etc. I use Home Assistant, so the bonus is then I can have those models fire and tie them to sensors that I can then automate (e.g. "If Gate is open and time is after sundown, turn on back yard lights and send me a notification with picture")
|
|
|
|
|
|
Why not look at YOLOv5 it is more advanced. I think DeepStack was ported from YOLOv5 V3.1 and currently YOLOv5 is at V6.1. If you do go with YOLOv5 the DeepStack models will need to be retrained.
|
|
|
|
|
That's absolutely where we're headed. DeepStack was an easy integration but it does have a lot of legacy baggage we want to clean out, and our feeling is simply replacing with the original code that they themselves used is probably the path of least pain.
cheers
Chris Maunder
|
|
|
|
|
I did find a package dataset, it is a small dataset, it only has 250 images. I am not sure how well it will train if I add it to one of my dataset because the datasets I am using for my custom models have over 10,000 images
Packages Object Detection Dataset - augmented-v1[^]
|
|
|
|
|
It's always worth trying out what we can, and then we make it better over time.
cheers
Chris Maunder
|
|
|
|
|
Hi Mike,
I noticed that Blue Iris sometimes flags objects as "cars" while other time as a "vehicle" - is vehicle a value label in your model? Or is this because "Mark As Vehicle" is enabled and thus Blue Iris groups multiple tags together under the vehicle label?
UPDATE: Nevermind, just saw there is a new Github page for the Sense AI versions of the models:
CodeProject.AI-Custom-IPcam-Models
IPcam-combined Labels: - person, bicycle, car, motorcycle, bus, truck, bird, cat, dog, horse, sheep, cow, bear, deer, rabbit, raccoon, fox, skunk, squirrel, pig
IPcam-general Labels (includes dark models images): - person, vehicle
IPcam-animal Labels: - bird, cat, dog, horse, sheep, cow, bear, deer, rabbit, raccoon, fox, skunk, squirrel, pig
IPcam-dark Labels: - Bicycle, Bus, Car, Cat, Dog, Motorcycle, Person
modified 8-Oct-22 15:05pm.
|
|
|
|
|
I was just going to post the exact same model details.
|
|
|
|
|