Financial services has a data problem: How AI is fueling innovation
In financial services, data is the lifeblood of every decision, trade, and report. Yet, the industry still relies on legacy infrastructure, siloed systems, and non-standardized data practices. As regulatory pressures increase, AI use accelerates, and demand for personalization intensifies, existing frameworks will reach a breaking point.
New industry-specific AI and data startups are emerging to bridge these gaps and redefine operational standards across financial services.
Financial services faces pervasive data challenges
Data complexity
Financial institutions (FIs) rely on an extensive range of data sources – market data, transaction records, client information, portfolio holdings, etc. — all in varied formats and structures. This variability, coupled with strict data governance and privacy requirements, complicates data management.
“The cost and complexity associated with managing diverse data across the organization is overwhelming.”
— Ralph H. Groce III, Former CIO at Wells Fargo
Financial data often relies on complex relationships and interdependencies; shifts in one data point can unpredictably impact others. For example, an interest rate change can simultaneously impact bonds, currencies, stocks, and commodities, requiring sophisticated models to capture holistic effects.
Data volume
The amount of financial data generated worldwide is staggering, with capital markets producing vast volumes of transactional, market, and client data every second. The NASDAQ sees 35+ million trades per day, and Visa averaged 700M transactions per day in 2023.
Regulatory regimes like MIFID II require banks, asset managers, hedge funds, etc., to report 65 data fields for nearly every equity, fixed income, and derivative trade (137B derivatives contracts alone were traded in 2023). These added processes necessitate efficient retrieval, formatting, and storage of massive amounts of data.
Legacy infrastructure
Many FIs have been operating for decades and rely on legacy systems. These technologies are often deeply integrated into operations and can be costly to replace. Many firms made incremental updates to infrastructure instead of full replacements, creating inefficient data flows and architecture. As explained to Insight by the Former Chief Data Architect at J.P. Morgan: “…that’s hundreds of [data transformation] steps and hundreds of systems that have been building up for 70 years…the likelihood of something going wrong is very high.”
Data silos
Organizational silos emerge as distinct business units have unique data needs. Based on our conversations with the Former CTO Architect for Global Risk and Control at Citi, “Each line of business is building their own systems which has led to this hodgepodge mess, and that’s the only way to put it… every large bank has this problem.”
As a result, data becomes challenging to locate across an organization. “AI/ML data scientists were spending an inordinate amount of time, something like 60% to 70%, just trying to find the right data,” explained the Former CTO for Data and Analytics at J.P. Morgan.
This can delay analysis, increase errors, duplicate work, and limit an organization’s ability to address emerging opportunities and risks.
Reconciling internal data can be challenging and adding additional counterparties, stakeholders, and data vendors makes the task nearly unmanageable.
“Reconciliation faces critical challenges, with fragmented systems and data inconsistencies threatening operational efficiency and compliance in an era of soaring transaction volumes and tightening regulatory demands.”
— Aman Thind, Chief Architect at State Street
The industry is reaching a breaking point catalyzed by regulation, AI, and personalization
Increased level of regulatory scrutiny
Recent events like the 2023 banking failures, rapid interest rate hikes, and heightened geopolitical tensions have amplified economic risk, pushing regulators to apply greater oversight. Regulatory bodies are increasingly data-driven, elevating expectations for real-time compliance and detailed data disclosures.
The stakes for non-compliance are high, demonstrated by the CFTC’s $200 million fine on J.P. Morgan for misreporting critical data fields in May 2024. Aggregate regulatory fines are also at historic levels. The SEC issued nearly $5B in enforcement actions in 2023, second only to 2022’s record-setting $6B.
Additionally, the scope of regulatory requirements is expanding. MIFID III (planned for 2025/2026) is expected to significantly expand reporting requirements, and operators are directly feeling these compliance pressures.
Real demand for AI
AI-powered models — from credit scoring algorithms to fraud detection systems — are only as effective as the data feeding them. As AI use expands, many firms are realizing their existing data infrastructure is not prepared.
“We are starting to see firms prioritize large data transformations to harness the power of AI. They are putting plans in place and increasingly looking to outsource data services.”
— John Costigan, Chief Data Officer at FactSet
Poor data hygiene — such as incomplete or inconsistent data — can compromise AI model performance. This gives way to severe consequences in financial contexts, where accuracy is critical.
“Your data must be AI-ready. Raw numbers lack meaning without context—tagging and maintaining high data quality are crucial to enabling AI.”
— Srinivas Gunturu, Director of Engineering Enterprise Data Delivery at Bloomberg*
AI is increasingly used for real-time decision-making. However, legacy systems weren’t designed to support such continuous, high-speed data flow. As a result, the gap between the demands of AI models and the capabilities of existing infrastructure is widening.
Heightened expectations for personalization and risk awareness
Financial clients, investors, and limited partners (LPs) increasingly expect portfolios and products tailored to their specific goals, risk profiles, and values. This is driven, in part, by increasing awareness of risk factors like ESG exposure, interest rates, and geopolitical events.
“Clients now want specific factors and sectors in their portfolio construction, which impacts everything from research to risk models and data. There are hundreds of different parameters to consider; platforms must be both highly adaptive to client needs and dynamic enough to consolidate complex information into a single interface.”
— Kfir Godrich, Chief Innovation Officer at BlackRock
ESG risk factors have become a central focus, with 89% of investors now considering ESG criteria in their investment decisions. Isolating specific risks is daunting given the multi-layered nature of financial instruments. Mutual funds and ETFs often comprise hundreds of underlying holdings, making accurate calculations dependent on evaluating each security’s ESG rating and aggregating the results based on portfolio weight.
Companies reshaping the landscape
Front Office
The front office includes “client-facing” activities such as trading, investment diligence, and wealth management, where revenue generation and client relationship management are key.
In recent years, several startups have emerged to streamline search and knowledge management, leveraging AI for front-office research, analysis, and decision-making. Hebbia, Rogo, and Sibli help enable banks, asset managers, and PE firms to analyze unstructured data quickly.
Auquan automates tasks like due diligence and portfolio monitoring by aggregating data from management accounts and public filings, giving FIs real-time views of business risks. “We use finance-specific RAG agents to automate complex diligence and monitoring workflows,” explains Founder and CEO Chandini Jain.
Reflexivity allows asset managers to analyze companies and test investment theses using natural language. “Mapping structured and unstructured data using our proprietary knowledge graph enables delivery of instant answers to deep computational questions,” says Cofounder and CEO Jan Szilagyi.
With 40% of FIs viewing front-office research as the top opportunity for AI, these tools have the potential to significantly reduce research hours and streamline analytical workflows.
Startups are launching to streamline front-office data analysis for PE/VC as well. Dealbase automates customer file analysis via “one-click outputs” in Excel. Quikirr helps simplify revenue analysis, data cuts, and KPI evaluation using AI for data reconciliation. “With a focus on deep Excel integration, we’re starting with common data room analyses and expanding to complete quantitative due diligence workflows,” says Quickirr Cofounder and CPO Mateo Varela.
Foresight combines deal sourcing and due diligence into a single platform, empowering private market investors to optimize investment decisions.
As AI tools improve, they will reshape how FIs handle research, analysis, and decision-making across the deal lifecycle.
Middle Office
The middle office bridges the front and back offices, ensuring necessary data, documents, and policies are in place to support workflows like portfolio monitoring. EdgeConnect uses AI to reconcile data from company systems of record (e.g., ERP/CRM) into a single, standardized view. “We combine PE expertise with software innovation to transform portfolio management and company reporting,” explains Founder and CEO Polina Kyriushko.
Shelton uses AI to provide investors with real-time portfolio analytics, which is critical for capital pacing, performance attribution, and allocation strategies.
Data management platforms offer versatile solutions, supporting everything from cleaning data to regulatory reporting. Arch provides “LPs, advisors, and treasury teams with a centralized hub to collaborate and access portfolio data at both the LP and asset levels. From small family offices to large private banks, Arch helps unite stakeholders on a single, integrated platform,” says Cofounder and CEO Ryan Eisenman.
Automated-Data leverages AI to interpret the semantic meaning of data, simplifying data integration and reconciliation. OutcomeCatalyst streamlines the preparation of complex alternative datasets, ensuring seamless integration with other data across the organization. KOR’s platform helps firms maintain compliance by increasing data transparency within transaction reporting. “Our mission is to bring an excellent user experience to achieving compliance through accuracy, completeness, and timeliness,” says KOR Founder and CEO Jonathan Thursby.
Back Office
The back office handles essential operational and administrative tasks, including transaction processing, trade settlement, record-keeping, and accounting. While incumbents typically focus on pure service delivery, 4Pines and TrustServe are tech-enabled fund administrators focused on alternative investors. 4Pines enhances accounting workflows for PE firms. TrustServe’s solutions cater to a broader range of alternative investors, spanning hedge funds, private equity, real estate, and family offices — with a focus on reporting and compliance.“We are a team of experts delivering high-quality professional services via cutting-edge technology to afford GPs greater focus on fundraising and deal-making,” explains TrustServe Founder and CEO Jorge de Cardenas.
These approaches combine technology with personalized service, modernizing and improving the offering while preserving a hands-on touch.
Software vendors are also emerging to centralize historically ad-hoc finance workflows. By centralizing capital calls, distributions, and investor reporting, Maybern helps finance teams reduce manual work and improve operations. Bunch helps fund managers and private investors set up, manage, and monitor investments. LemonEdge offers a robust, end-to-end accounting system for private markets, automating multi-ledger management and event-driven transactions.
Together, these tech-enabled services and software solutions are bringing greater efficiency and centralization to an increasingly complex office of the CFO.
Advice for founders
The need for innovative AI development, combined with changing regulatory demands, has fostered an environment where agile startups are positioned to succeed. FIs are showing genuine interest in adopting AI-powered tools.
“It’s almost impossible to resist going to AI. The productivity improvements are real, and there’s genuine interest at the board level across the industry.”
— George Stein, Director of Compliance and Operational Risk at Bank of America
That said, targeting the right FI is vital for startups. Founders should target institutions that are small enough to favor outsourcing but large enough to encounter operational complexity that requires external solutions. Focusing on signals of adoption is also key, as organizational structures and strategic hires, such as a “Chief AI Officer,” indicate a company’s readiness to embrace emerging technologies.
Startups must understand the key risks that concern buyers.
“Data leakage risk is something CISOs care about. Is any of our data leaking such that a model is being trained on confidential data?”
— Vibhor Rastogi, Global Head of AI Investments at Citi
Additionally, founders should consider prioritizing complementing existing systems:
“Startups should target capabilities that extend investments already made in hyperscalers. It is more compelling to buy when we can enhance existing, entrenched infrastructure rather than replace it.”
— Kjersten Moody, Global Chief Data Officer at Prudential Financial
Through intentional go-to-market strategies and product risk mitigants, startups can capitalize on technological and regulatory tailwinds present in today’s market.
If you’re building in the space, we’d love to meet! Email us at earnowitz@insightpartners.com.
Editor’s Note: Insight Partners has invested in Duco.
Quotes in this article have been sourced from industry conversations and given with permission.
*Srinivas Gunturu has indicated all opinions are his own.
