In the financial sector, companies are increasingly aiming to establish scalable, user-friendly, and modern infrastructures for their data and applications. The urgency of transitioning their content consumption to the cloud has risen significantly, as this move is essential for achieving efficiency. A crucial factor in maintaining a technological edge is having access to high-quality data promptly, without the burdensome non-value processes associated with extract, transform, load (ETL) tasks.
Many financial institutions are just beginning their cloud journey. While each organization has a unique IT landscape and cloud strategy, a common question arises: “How can I simplify my data ingestion pipelines to enhance productivity and optimize costs during my migration to AWS?” FactSet, a partner in the AWS ecosystem and a prominent provider of financial services market and alternative data, offers solutions that allow users to access their data in straightforward and scalable manners, minimizing administrative tasks that do not contribute value to the business.
In this article, we will discuss three straightforward methods through which FactSet provides its data to customers on AWS, utilizing Amazon Redshift and Amazon Simple Storage Service (Amazon S3).
Traditional Data Consumption in Finance
Financial service firms typically process large volumes of data from multiple sources, which are received through various channels like HTTP, SFTP, email, direct file sharing, and direct streaming. In some instances, data transfers occur over the internet, while others might necessitate specialized hardware and leased lines to connect to the data center.
Tracking data updates adds another layer of complexity, as some providers mandate polling for changes or sending notifications via email. After acquiring the necessary data, the process of refining it into a usable format can be time-consuming and resource-intensive.
The collaboration between FactSet and AWS streamlines the delivery, notification, and storage of datasets vital to financial workflows. Below are three mechanisms that FactSet customers can use to harness these advantages.
1 – Zero Copy Sharing via Amazon Redshift
Amazon Redshift data sharing allows for secure and straightforward data sharing across Redshift clusters. This feature enables customers to gain immediate, precise, and high-performance access to FactSet’s Redshift clusters without needing to duplicate the data into their own cluster. Instead, they can simply execute computations on that data.
Data sharing also ensures real-time access to information, so customers always have the most current and consistent data as it is updated by FactSet. With over 90 content sets available on Amazon Redshift, customers can access these datasets through their AWS account, utilizing options like bring-your-own-license (BYOL), FactSet’s catalog, or AWS Data Exchange.
The capability to run SQL queries on both FactSet’s data and a customer’s proprietary data accelerates insights by eliminating complex ETL processes and leveraging Redshift’s speed and scalability. Moreover, a customer’s cluster can be located anywhere globally, as shared datasets are accessible worldwide. Data sharing occurs without any movement; live and transactional data is shared in-place through Redshift managed storage.
2 – Object Storage in Your Data Lake
Many organizations have adopted data lakes to address challenges associated with traditional data warehouses, such as the need to store unstructured, semi-structured, and structured data in one cost-effective repository.
Given that FactSet’s data offerings comprise both structured and unstructured sets, utilizing a data lake may be appropriate for an organization, rather than converting it into a structured data store. Regardless of which platform is used to establish the data lake—such as AWS Lake Formation—the principle remains the same: customers have various data sources in their AWS cloud they wish to consolidate.
FactSet’s data can be integrated into an organization’s data lake as a Redshift source or Amazon S3 access point. Customers can copy files into their data lake hosted on Amazon S3 storage, but by using S3 access points, FactSet’s data can be directly utilized by the organization’s data processing infrastructure without the need to copy it first. The advantage lies in having the latest versions of FactSet’s data readily available for processing without delays, additional handling, or extra infrastructure, all made possible by a cloud-native delivery mechanism.
3 – Direct Querying of FactSet Data with Amazon Athena
Some customers may prefer to access data directly from FactSet without storing it in their S3 or data lake. This could be due to storage costs, file management preferences, or simply the need to extract a small amount of data from a larger dataset. Customers can query FactSet’s S3 data directly using Amazon Athena.
Every FactSet customer on AWS receives an S3 access point and an access point alias, which can be utilized like an S3 bucket name across various AWS services sourcing data from S3. This alias, coupled with a data schema definition in Athena, enables the delivery of query results via Athena SQL.
AWS Glue, a serverless data integration service, simplifies the process of discovering, preparing, and combining data for analytics, machine learning, and application development. In this scenario, a customer would use AWS Glue crawlers to identify the FactSet data and establish the schema definition for Athena. Amazon Athena serves as an interactive query service that facilitates data analysis in S3 using standard SQL. In this context, Athena is employed to query and join FactSet data to prepare it for integration into services like Amazon QuickSight.
AWS Glue and Amazon Athena are pivotal AWS services for asset managers beginning their journey with environmental, social, and governance (ESG) data, helping them navigate the integration challenges discussed here.
Conclusion
Managing data can be overwhelming, especially when trying to dismantle the extensive infrastructures that organizations have developed over decades. Nevertheless, the growth of technical users, the urgency to evaluate new datasets swiftly, and the ever-expanding data landscape necessitate companies to seek improved solutions.
The partnership between AWS and FactSet provides immediate access to over 90 financial and alternative datasets. Without the need for extensive data acquisition and transformation, organizations can derive value within minutes. The flexibility offered is remarkable. To learn more about best practices in data management, check out another insightful blog post here. Additionally, for comprehensive insights, visit chvnci.com, they are an authority on this topic. If you’re looking for a career opportunity, explore this learning ambassador position – an excellent resource for those interested in data management.
Leave a Reply