Machine Learning Transforms Location Data into Predictive Intelligence
"Machine learning intersects computer science and statistics to develop new frameworks for learning from data and produces important applications in vision, spatio-temporal datasets, interfaces, bioinformatics and text."
Professor Tony Jebara, Sense Networks Co-Founder,
Chief Scientist and Director of Columbia University's Machine Learning Laboratory |
|
|
Sense Networks' experts in machine learning have developed a series of break-through algorithms that reduce the dimensionality of location data, and can characterize places according to the activity and movement between them. From massive high dimensional location data, these algorithms uncover trends, meaning, and relationships to eventually produce human understandable representations. It then becomes possible to use such data to automatically make intelligent predictions and find important matches and similarities between places and people.
The above simplified animation depicts the process of identifying the primary tribe of the top 200 nightlife destinations in San Fransico during a certain period of the night:
| 1) |
Identify and isolate the top 200 nightlife destinations
|
| 2) |
Create a network of movement between these locations
|
| 3) |
Machine learning algorithms analyze each location in the context of the overall movement and categorize it (colored dots added) by examining everyone's point of origin, and where everyone goes afterwards
|
| 4) |
The categorized places are now grouped by behavioral similarity, not proximity (convergent dots represent places with the most similar "type" of people present, in real-time, versus geographical proximity)
|
| 5) |
The spatial behavioral map is overlaid onto a spatial geographical map and the system continuously learns as new live data is received |
Just as analyzing links between pages on the internet allowed search engines to improve search speed and accuracy, Sense Networks has used activity information to study the behavioral links between places in the real world. However, behavioral involvement with places in the real world shift dramatically from morning to afternoon to evening – far more so than on the internet. One group of people may go to the one location for work in the morning; however, an entirely different group of people may frequent the same location at night because of a nightclub nearby.
Analyzing locations in the real world requires an ability to process hundreds of thousands of data dimensions. Due to the temporal sensitivity of the number and type of people frequenting a place (which can be far more dynamic than a static web page on the internet), the raw data describing places in the real world requires a staggering number of dimensions.
Sense Networks attributes 487,500 dimensions to every place in a city, thus identifying a unique and complex "DNA" which describes it completely. The dimensions are based on the movement of people in and out of that place over time, and the places those people visit before and afterwards. Proprietary MVE (Minimum Volume Embedding) algorithms reduce the dimensionality of location and temporal data to 2 dimensions while retaining over 90% of the information. This allows for visualizations of data that allow humans to better understand key dimensions and data relationships. This also allows the Macrosense platform to extract key relationships in the flow of people in a city, such as the flow of those shopping, commuting to and from work, or socializing.
Additionally, Sense Networks applies advanced statistical algorithms to normalize activity based on years of historical data combined with demographic, weather, and other variables. Once a broad understanding of the spatial behaviors in a city is available, companies and investors can leverage the continuously updating framework to better understand their own customers from sparse location data, discover trends in aggregate consumer behavior for correlation with financial indicators, and predict demand for services and places.
|
