Crime scenes are complex and often contain multiple components. As such, they require an in-depth study to catch criminals who may have committed the crime. Microsoft has invented a system to perform automated crime scene analysis using machine learning. The system is nothing but “Sherlock Holmes” bot, as it examines the crime scene as meticulously as Sherlock Holmes.

A recent patent application from Microsoft reveals details of the “Sherlock Holmes” bot. The figure below (from the patent application) shows a crime scene, including a number of components. To an untrained eye, the footprints and the body will be the only points of interest, when in fact, all the objects in the crime scene (for. e.g. furniture, TV, walls) will play a role in the analysis of the crime scene. If the crime includes discharge of a firearm, each object in the crime scene affects the impact of the bullet. What’s more, stray or misfired bullets could also be lodged in any of the objects, like the walls.

The system will capture data related to crime scenes from sensory devices including cameras, microphones, electronic and alarm systems to perform analysis of the crime scene using machine learning algorithms. For example, the layout of the crime scene will be deduced. The layout will also impact the data captured by the sensory devices. For instance, if a gunshot is captured in a microphone, the presence of highly reflective and large furniture pieces will affect the sound recorded by the microphone. This distortive effect will be taken into account while determining the type of firearm used in the crime.

The machine learning algorithms will be trained based on pre-recorded audio and visual information associated with crime scenes, like information associated with firearms. The machine learning algorithm will then improve over time after consuming new data related to crime scenes that is analyzed using the machine learning algorithms. This will result in a system that grows smarter with additional information and provides a more in-depth analysis. For example, the system will recognize a type of crime scene and determine types of weapons generally used in the setting. The system will also be able to describe the impact of bullets on the victim by sound and visual and be able to tell if the bullet may hit a bulletproof vest.

The system will then, as Mr. Holmes, be able to provide the identification of a weapon, a comprehensive crime scene analysis report (describing how all components in the crime scene may have affected the criminal activity), and a feedback report to designated recipients (like Inspector Lestrade). All this in-depth analysis will definitely make solving crimes “elementary” for law enforcement agents.

And if Microsoft partners up with IBM to implement this system (to improve its machine learning model), law enforcement agencies may soon be taking the aid of the “Sherlock Holmes” bot from Microsoft, and Dr. Watson® from IBM.

US20190057286A1