Apac
  • Home
  • CXO Insights
  • CIO Speaks
  • Partner Conferences
  • Newsletter
  • Subscribe
  • News
  • About us
Apac
  • Admired Tech

    Agile

    AI Healthcare

    Artificial Intelligence

    Augmented Reality

    Aviation

    Big Data

    Blockchain

    Cloud

    Cyber Security

    DevOps

    Digital Transformation

    Drone

    HPC

    Internet of Things

    IT Services

    Marine Tech

    Networking

    Plastic Tech

    PropTech

    Robotics

    Sensor Tech

    Simulation

    Smart City

    Software Testing

    Startup

    Storage

    Unified Communication

    Web Development

    Wireless

  • Automotive

    Banking

    Capital Market

    Compliance

    Construction

    Custom Software Development

    E-Commerce

    Education

    FinTech

    Food and Beverages

    Gov and Public

    Healthcare

    Insurance

    Legal

    Logistics

    Manufacturing

    Media and Entertainment

    Metals and Mining

    Pharma and Life Science

    Retail

    Sports

    Travel and Hospitality

  • CISCO

    Google

    IBM

    Microsoft

    Oracle

    Salesforce

    SAP

    ServiceNow

  • Business Intelligence

    CEM

    Cognitive

    Collaboration

    Contact Center

    Corporate Finance

    CRM

    Data Center

    Digital Signage

    Enterprise Architecture

    Enterprise Asset Management

    Enterprise Communications

    Enterprise Performance Management

    ERP

    Facility Management

    Field Service

    Fleet Management

    Gamification

    HR Technology

    IT Infrastructure

    IT Service Management

    Managed Services

    PLM

    Procurement

    Project Management

    RegTech

    Revenue Management

    Sales Tech

Menu
    • Amazon
    • Cloud
    • HPC
    • Capital Market
    • Collaboration
    • CEM
    • Drone
    • Facility Management
    • Fleet Management
    • Food and Beverages
    • Business Intelligence
    • IBM
    • Insurance
    • IT Infrastructure
    • Legal
    • Marine Tech
    • Networking
    • Plastic Tech
    • Procurement
    • RegTech
    • Sensor Tech
    • Simulation
    More
    Facility Management Fleet Management Food and Beverages Business Intelligence IBM Insurance IT Infrastructure Legal Marine Tech Networking Plastic Tech Procurement RegTech Sensor Tech Simulation
    #

    Apac CIO Outlook Weekly Brief

    ×

    Be first to read the latest tech news, Industry Leader's Insights, and CIO interviews of medium and large enterprises exclusively from Apac CIO Outlook

    Subscribe

    loading

    THANK YOU FOR SUBSCRIBING

    • Home
    • IT Services
    Editor's Pick (1 - 4 of 8)
    left
    What Does it Take to be a Banker of Tomorrow?

    Osman Faiz, Chief Information Officer, Singapore, Standard Chartered Bank

    Establishing a Strong, Future - Ready IT Infrastructure

    Voranuch Dejakaisaya, CIO, Bank of Ayudhya (Krungsri)

    CIOs must Ensure Systems are Dynamic, Accessible and Continuous

    Zeeshan Sheikh, CIO, Entergy Corporation

    Transitioning from CIO to CDO

    Bruce Coller, CIO, Sir Moses Montefiore Jewish Home

    Adoption of Smart Technology for Responsible Gaming

    Yeo Teck Guan, Group CIO, Singapore Pools

    What Digital means to a CIO

    Geoff Wenborn,

    A 360 View in SDN/NFV for Smarter Transformation

    Manjeet Dhariwal,

    Business Leaders Should Be Selective About the Problems Undertaken

    Todd Gottula, CTO, Advent Software

    right

    Big data analytics: Tackling the historical data challenge

    By Simon Garland, Chief Strategist, Kx Systems

    Tweet
    content-image

    Simon Garland, Chief Strategist, Kx Systems

    Big data is noisy, messy and, frequently, inconsistent. To make sense of it, companies are investing time and money in an array of technology solutions. However, some much-touted approaches are actually quite costly and in many ways, have not yet addressed the expanding set of historical information that companies want to mine.

    A solution to these shortcomings is to take a hybrid approach – combining the performance and speed of in-memory processing with the vast data storage capabilities that a traditional on-disk approach offers.

    The challenge: speed versus space

    To get meaningful results from big data, companies need to slice and dice their data in many different ways to work out which parts are worth using. Considering the enormity of the task, for many, this requires– above all – speed.

    One approach to dealing with this data is in-memory computing. This delivers both performance and speed, allowing companies to analyze data dynamically and quickly. For many companies, however, it is not always the most feasible solution. This is because of the high cost of keeping and running huge data sets in-memory.

    “Combine the performance and speed of in-memory processing with the vast data storage capabilities that a traditional on-disk approach offers”

    While hardware makers have been building machines with more and more memory, even when the memory and storage problems are solved, those using more traditional database approaches to work with historical data will often struggle with speed at scale.

    A hybrid solution

    For years, the financial sector has successfully combined inmemory processing with on-disk storage to manage trade data, which adds up to billions of records per day, in a fast, yet efficient manner. Other industries facing similar data can learn from their example.

    With a more hybrid approach, companies can combine the high performance and speed capabilities of in-memory while solving the storage issues by putting the vast historical data sets on disk. By bridging available technologies, companies can deliver on all counts – including cost.

    Crucially, by folding in a high performance programming language right in with the data, users can interact directly with their data in one place. This gives a super-charged in-memory and on-disk historical database at their fingertips and the ability to deliver results in speeds and complexity previously unavailable. In a world where the most interesting information is also the trickiest to manage, this is without doubt the Holy Grail of data management.

    See Also:
    Top Big Data Solution Companies
    Top Big Data Consulting Companies
    tag

    Big Data

    Read Also

    4 Reasons Why Digital Transformation should be a Top Down process

    4 Reasons Why Digital Transformation should be a Top Down process

    Because Strategies and Security Goes Hand-In-Hand!

    Because Strategies and Security Goes Hand-In-Hand!

    Digital Opportunities in the Asia Pacific are Ready to Explode

    Digital Opportunities in the Asia Pacific are Ready to Explode

    Pitfalls to Avoid When Implementing MDM

    Pitfalls to Avoid When Implementing MDM

    Weekly Brief

    loading
    Top 10 IT Service Companies - 2019

    Featured Vendors

    Wyntec

    Tom Freer, MD

    Basis Bay

    Praba Thiagarajah, CEO

    ON THE DECK

    IT Services 2019

    Top Vendors

    IT Services 2018

    Top Vendors

    IT Services 2017

    Top Vendors

    IT Services 2015

    Top Vendors

    Previous Next

    Copyright © 2019 APAC CIOoutlook. All rights reserved. Registration on or use of this site constitutes acceptance of our Terms of Use and Privacy Policy  |  Sitemap

    follow on linkedinfollow on twitter follow on rss
    This content is copyright protected

    However, if you would like to share the information in this article, you may use the link below:

    https://it-services.apacciooutlook.com/cxoinsights/big-data-analytics-tackling-the-historical-data-challenge-nwid-413.html