Big Data Analytics

Extract, Transforming & Loading (ETL)

Big Data Analytics

Sphere Software specializes in designing and building custom back end systems designed to handle todays data-heavy business functions. Our engineer have expertise creating systems to handle the most complex enterprise level data analytics processes.

Big Data

Got a big data project?

Let us put our expertise to work for your model to create scalable architecture that provides granular reporting capabilities and real-time data visualization tools.

We have over ten years of experience in all leading database and business intelligence technologies:

BDA Technologies

We carry extensive experience in all of the data-heavy industries:
  • eCommerce & Marketplace Technology
  • Financial Technology (FinTech)
  • Healthcare Technology (HealthTech)
  • Advertising Technology (AdTech)
  • Marketing Technology (MarketingTech)
  • Energy Technology (EnergyTech)

Want to talk to one of our database analytics experts?

Big Data Analytics Customer Case Study

Sears Sears
Technologies
  • PHP
  • Java
  • Node.js
  • Redis
  • MongoDB
  • Apache Kafka Messaging
Solution Type
  • Solution Architecture
  • Custom Software Development
  • Extract, Transform & Load (ETL)
  • Team Augmentation
Problem

Sears is one of the leading retail brands in the country pioneering wearable or smart sensor technologies. As these new innovations have become increasingly popular, so has the need for adequate storage and processing capacities.

Approach

Sphere continues to provide several database developers on an ongoing basis. These developers provide business intelligence, architectural engineers, data modeling and advanced visualization features that allow all stakeholders to communicate on company performance in real-time.

Extract, Transforming & Loading (ETL)

Sphere has a great deal of experience solving a wide range of database challenges across all of the data-centric industries of finance, healthcare, eCommerce and digital advertising & marketing. We’ve helped countless clients extract, sort and convert hard to handle data from their legacy systems into a new, more workable format. We then loaded that new data into a custom designed database application that provided enhanced data analytics and visualization features.

Solution Architecture

Considering an ETL project?

ETL Customer Case Studies

Lend'sEnd Lend'sEnd
Technologies
  • Netezza
  • J2EE
  • Struts
  • Hibernate
Solution
  • Solution Architecture
  • Custom Software Development
  • Extract, Transform & Load (ETL)
  • Team Augmentation
Problem

Lands’End was trying to find a more effective way to extract actionable insights from terabytes of customer data. Their massive operational database lacked the functionalities required by their marketing team to produce the analytics needed to reach new customer segments.

Response

Sphere built a ETL process to extract data from the Lands’End operational database. A custom built web application was then created to manage all this extracted data for use by the Lands’End marketing team.

CORPORATE & INVESTMENT BANKING
CORPORATE & INVESTMENT BANKING
Technologies
  • Scala
  • Spring Integration
  • Pivotal Gemfire
  • RabbitMQ
  • Oracle
  • C24
Solution
  • Solution Architecture
  • Software Development
  • ETL ( Extract, Transforming, & Loading )
Problem

Societe Generale (SG) has nearly 12,000 employees in 37 countries who deliver worldwide expertise in investment banking, global finance, and global markets. SG has a complex distribution system for processing trading instructions. Although these instructions originate from exchanges, they are routed to the appropriate subsystems for further processing. During the routing process, these instructions are enhanced by applying routing and enrichment rules and then translated to different formats. These rules often change due to modifications in financial regulation. Sphere was asked to change the message flow routing which involved the OCC Exchange. The OCC is the registered clearing facility for all U.S. exchange-listed securities options. Sphere was also required to coordinate the message processing in multiple data centers and subsystems in different geographical areas.

Solution

​Sphere addressed these challenges by developing new trading message flows between the required exchanges and financial systems with Scala and Java. The routing rules and adapter pipelines for message enrichment and validation were changed in the existing NVision framework. Sphere formatted and executed the field transformation for messages in NewClear which is a subsystem that reconciled the buying and selling of trading, as well as for GMI and Shadow. Routing rules, enrichment data, and configuration information were obtained via GemFire and Oracle servers. Sphere also developed and implemented a unique message broker substitution workaround for the UAT ( User Acceptance Testing ) environment which simulated various infrastructures for message flows. Lastly, flow visualization in NVision Web Dashboard was also adjusted.