Development of an Advanced Data Pipeline for Custom Financial Index Creation
A secure, high-speed, and scalable data ingestion and processing system for financial indexes, capable of handling data from multiple sources including CME, Bloomberg, and Reuters, with real-time and batch processing capabilities, data validation, and automated backup and recovery features.
Python for data processing and pipeline development, Apache Beam for scalable and flexible data processing workflows, Google Cloud Platform services including Pub/Sub, BigQuery, and Cloud Storage...