Gathering insights from big data regarding future trends requires data scientists’ expertise and can take weeks or even months. Now, an MIT spinout has developed a predictive analytics platform that lets anyone upload raw data and input any question into an interface — similar to using an online search engine — and receive accurate answers in just 15 minutes.
The platform, developed by Endor, has useful applications in the business and defense sectors, in response to questions such as which customers want certain services, where to open new stores, predicting traffic jams, or detecting potential terrorists.
The platform is based on the science of “social physics,” co-developed at the MIT Media Lab by Endor co-founders Pentland, (the Toshiba Professor of Media Arts and Sciences), and a former MIT postdoc. Social physics uses mathematic models and machine learning to understand and predict crowd behaviors.
At an early stage, the platform was tested with the U.S. Defense Advanced Research Project Agency (DARPA) to analyze mobile data in certain cities in times of civil unrest to show how emerging patterns can help predict future riots.
Users of the new platform upload data about customers or other individuals, such as records of mobile phone calls, credit card purchases, or web activity. The platform identifies patterns of previous behavior among the data and uses social physics models to predict future behavior. The platform can also analyze fully encrypted data-streams, allowing customers such as banks or credit card operators to maintain data privacy.
The project is financially backed by Innovation Endeavors, the private venture capital firm of the executive chairman of Google parent company Alphabet, Inc.
Recently, Endor analyzed Twitter data for a defense agency to detect potential terrorists. The company was given 15 million data points containing examples of 50 Twitter accounts of identified ISIS activists, based on identifiers in the metadata. From that, they asked the startup to detect 74 with identifiers extremely well hidden in the metadata. Someone at Endor completed the task on a laptop in 24 minutes, detecting 80 “lookalike” ISIS accounts, 45 of which were from the pool of 74 well-hidden accounts named by the agency.
Endor isn’t aimed at replacing data scientists; it’s designed as a tool to empower them. Data scientists are most familiar with their organization’s business semantics and can incorporate Endor into their workflow. By opening a “bottleneck” — where data input comes in faster than anyone can produce an output — the project aims to help data scientists improve their companies.
The forthcoming Big Data Program will focus on aspects and applications of big data in the security and homeland security spheres: Analytics, intelligence for action, prediction, sensing, cyber threats, integration, and more. (Ramona E.).
