Click here to Skip to main content
15,867,568 members
Articles / Artificial Intelligence / Tensorflow

Alerting the Driver with AI Hazard Detection on Android

Rate me:
Please Sign up or sign in to vote.
5.00/5 (2 votes)
8 Jan 2021CPOL3 min read 5K   2  
In this next entry in the AI Hazard Detection on Android series, we will look at how we can avoid unnecessarily warning the user, and how we can quickly alert them.
Here we will make an opt-in list of objects that should generate alerts, and look at preventing alerts from being sent when the driver is not moving.

This is the fourth entry in a series of articles on building a real-time hazard detector using Android. In the previous article, we looked at how to highlight objects on the screen. We also noted that not every object detected is something we want to alert the user to. Other objects that the detector recognizes include birds and other vehicles, neither of which is something for which we want to alert the user. The detector does recognize people and various animals. We would want alerts for these. If the car is not moving then it may be unnecessary to alert the user at all; people walking across a street while a car waits is one scenario where a recognized entity enters the phone’s field-of-view, but a scenario in which we would want the system to not generate an alert.

To avoid unnecessarily alerting a user, we will make an opt-in list of objects that should generate alerts. Alongside the TensorFlow Lite model and the labels file, I have added a new file named concerns.txt. Each line of concerns.txt contains a line copied from labels.txt that is a label for something for which the detector will issue alerts. The detector loads these into a list in the same way that it loads the labels.

Python
if(concernList.size() == 0) {
   InputStream concernStream = context.getAssets().open("concerns.txt");
   BufferedReader br = new BufferedReader(new InputStreamReader(concernStream));
   String line;
   while(null != (line = br.readLine())) {
       Log.i(TAG, String.format("Concern: %s", line));
       concernList.add(line);
   }
}

Sound is one option for alerting the user. If a device has stereo speakers, we can take advantage of them by generating sounds that play through the left speaker, right speaker, or both speakers to quickly indicate a general direction of the hazard. To accomplish this, I’ve made three sound files and have added them to the project’s resources. These sound files are placed in the folder app/src/res/raw. The sound files are added to the resources that we can refer to in code. I defined a few constants that hold the IDs of the sound files to reference them more easily. A variable to refer to a MediaPlayer is also needed. The MediaPlayer will be responsible for playing sound.

val ALARM_LEFT: Int = R.raw.left_sound
val ALARM_RIGHT: Int = R.raw.right_sound
val ALARM_CENTER: Int = R.raw.center_sound
var soundPlayer: MediaPlayer? = null

Creating a MediaPlayer object with one of the sound values gives us an object that can play our alert sound.

Preventing Continuous Alerts

To prevent alerts from being sent when the driver is not moving, there is a minimum speed that the user must exceed. If the user is below this speed, no sounds alerts are generated. Android’s location services return speed in meters per second. When thinking of driving speeds, I tend to think more in kilometers per hour. To set the minimum speed, I have defined one constant that sets the minimum kilometers per hour, and another constant uses this first constant to express the same speed in meters per second.

Python
val MIN_ALERT_KMPH:Float = 10.0f
val  MIN_ALERT_MPS:Float = MIN_ALERT_KMPH*1000.0f/(60.0f*60.0f)

The detector will receive updates on the current speed of the vehicle. For the test interface, which is made for debugging and testing, the speed will be set in code and not dependent on the actual movement of the vehicle.

Rather than alert a user continuously about a hazard, there is a cooldown time before another alert is issued. When we attempt to sound an alarm, we first check whether there has been an alarm that hasn’t expired yet. If the expiration time has not passed then the alarm will not sound. Attempting to sound the alarm will also reset the cooldown time.

Python
var coolDownExpiry:Long = 0
val COOL_DOWN_TIME = 10000

fun resetCooldown() {
   coolDownExpiry = Date().time + COOL_DOWN_TIME
}
fun hasCooldownExpired():Boolean {
   val now = Date().time
   return now > coolDownExpiry
}



fun alert(direction: Int) {
   // using the when statement to filter out invalid
   // values should they be passed to this function
   if(hasCooldownExpired() && currentSpeedMPS>MIN_ALERT_MPS) {
       when (direction) {
           ALARM_CENTER, ALARM_RIGHT, ALARM_LEFT -> {
               soundPlayer = MediaPlayer.create(context, direction)
               soundPlayer!!.start()
           }
       }
   }
   resetCooldown()
}

Most of the needed functionality for the application is now in place. The last few things to add are crash detection with emergency alerts and having the application use the live camera stream instead of user-selected images. In the next entry of this series, we will look at crash detection and notifications.

This article is part of the series 'Realtime Hazard Detection on Android with Tensorflow Lite View All

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)


Written By
Software Developer
United States United States
I attended Southern Polytechnic State University and earned a Bachelors of Science in Computer Science and later returned to earn a Masters of Science in Software Engineering. I've largely developed solutions that are based on a mix of Microsoft technologies with open source technologies mixed in. I've got an interest in astronomy and you'll see that interest overflow into some of my code project articles from time to time.



Twitter:@j2inet

Instagram: j2inet


Comments and Discussions

 
-- There are no messages in this forum --