Big Data Solutions Emerge

In-memory databases and massively parallel processors are deployed.



Capital markets firms are coming to grips with the challenges associated with capturing, storing and analyzing enormous volume of data in a cost-efficient manner.



“We can answer most of our problems with existing data analytics, but what we find intriguing about Big Data is the cost structure of storing large amounts of historic data,” Ed Dabagian-Paul, vice president of technical architecture at Credit Suisse, said during a recent webinar.



Big Data projects are being undertaken in litigation response/regulatory compliance; control over internal data and applications, risk analytics/enterprise risk management; and trading analytics/on-demand database analytics.



“Capital markets institutions of all types-buy side, sell side, exchanges, regulators and service providers—are considering and in some cases implementing Big Data solutions in response to the huge volumes of data they are forced to deal with,” according to a report by A-Team Group.



While Big Data solutions are still in their infancy, market practitioners are taking their promise of handling large volumes of structured and unstructured data seriously, and expect to see deployment sin the next 12 to 18 months, the report said.



According to a survey of data technologists in financial institutions conducted by A-Team Group, few firms have solutions in place to handle unstructured data such as email, IMs, PDFs, audio and video, and social media.



Firms are attempting to synthesize traditional database technology for structured data with emerging Big Data technologies for unstructured data, such as Hadoop.



The survey found that massively parallel processors, which involve the coordinated processing of a program between multiple independent computers, each with its own operating system and memory, were cited by 31% of respondents as a potential solution.



In-memory databases, which store data in main memory rather than on disk, were cited by 17% of respondents, and NoSQL, which are “shell” relational database management systems that don’t use Structured Query Language, were cited by 15% of respondents.