What is best way of processing the large volume of data in OLTP application? Weather it should be”Near Data Processing” or “Far data Processing”? Process Near the data (database procedures, PLSQL, ETL, ELT, OOD etc) – Process away from the data ( Java coding, C++ coding, etc.) Both the systems has some pros and cons:
Processing Near Data
Generally two type of processing occur one is batch processing and other is real time processing. Batch processing where a group of data is being processed and entire processing is very similar, like salary calculation and disbursement, loyalty calculation, interest calculation and posting trading day end processing, settlement of bills, bill generation etc. Real time processing are those processing which occurs realtime like online fund transfer, money withdrawal, purchase of any item and it’s payment. Batch processing should be carried out near the data and real time should be carried out near the objects. While certain systems perform batch processing far from the data(like social networking sites) but those systems have some valid reasons of distributed processing and time does not matter to those systems like but there we time is crucial factor and window is very small it is preferred to move the processing near the data. Overhead of network, chances of data loss, disconnection of process, and availability of systems are main reasons to move the processing near data.
Processing Far Data
Far Large Volume of data processing is not very encouraging in the data analytical world. There are some cases where far data processing is beneficial like form processing, bulk entry(offline) processing, social networking sites data analysis for the presentation to users. It is also essential when system is processing individual transaction based out of bulk transaction processing far data processing is really good to have.